Using your own Hugging Face access token in CAS

NVIDIA NeMo Retriever Extraction, also known as nv-ingest, uses models that are downloaded from Hugging Face to do token-based splitting on the content extracted from documents. NVIDIA recommends to use the meta-llama/Llama-3.2-1B tokenizer, however this model is gated and requires a Hugging Face token in order access it. A Hugging Face access token is a personal API key that is used to securely authenticate programmatic access to Hugging Face services.

Before you begin

You must have a valid token to access Meta's Llama 3.2 language models and evals gated repository.

About this task

In Content-Aware Storage (CAS), a Hugging Face access token is used to securely interact with Hugging Face's APIs and services that support intelligent content processing.

For more information about Hugging Face access tokens, see Hugging Face User Access Tokens.

To provide your Hugging Face access token for the meta-llama/Llama-3.2-1B model, perform the following steps:

Procedure

Encode the generated Hugging Face access token in Base64 format by using the following command. The resulting value is refereed to as the HUGGING_FACE_ACCESS_TOKEN in the next step.
```
echo "hf_xxxxxxxxxxx" | base64
```

Create a secret in the CAS installation namespace:

kind: Secret
apiVersion: v1
metadata:
  name: hf-access-token
  namespace: ibm-cas
data:
  hf_access_token: HUGGING_FACE_ACCESS_TOKEN
type: Opaque

Use the encoded access token and update the cas-install CR 'spec' section:
```
spec:
  flags:
    - HF_ACCESS_TOKEN_SECRET_NAME=hf-access-token
```