Troubleshooting
Problem
You are using IBM Cloud Pak® for Data 5.0.2 in an air-gapped environment and are attempting to import a custom model (Bring Your Own Model [BYOM] feature) from Huggingface into your cluster. The model being attempted is Llama-3.1-Nemotron-70B-Instruct-HF and you are attempting to deploy it to a model pod with the following configuration:
- 8 GPUs
- 8 CPUs
- 600GB RAM
While launching the model pod, the following error occurs, preventing the model from running:
"SafetensorError: Error while deserializing header: MetadataIncompleteBuffer"
Due to the secured/air-gapped nature of this cluster, you can't download models from the huggingface directly. So you download the model and send it to the bastion node so that you can make the pvc with that. The safetensor files that has been transferred are symbolic links to the relevant blob files.
Symptom
The following error is observed when attempting to run/deploy the model:
"SafetensorError: Error while deserializing header: MetadataIncompleteBuffer"
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB10","label":"Data and AI"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSGU851","label":"IBM Watson Studio for IBM Cloud Pak for Data"},"ARM Category":[{"code":"a8m3p0000006xuZAAQ","label":"Services-\u003EData Science Tools-\u003Ewatonx.ai"}],"ARM Case Number":"TS018079809","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]
Log InLog in to view more of this document
This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.
Was this topic helpful?
Document Information
Modified date:
19 December 2024
UID
ibm17179593