Registering a custom foundation model

After creating a PVC and copying the custom foundation model into the PVC, a cluster administrator must register the custom foundation model to make it available for deployment and inferencing with watsonx.ai™.

Registering your model

To register a custom foundation model:

  1. Log in to OpenShift and then edit the Watsonxaiifm custom resource (CR) file. To register a new model you have to append a model entry in this file:
    oc edit Watsonxaiifm
  2. Add a model entry under spec.custom_foundation_models and enter the model details. See Properties and parameters for custom foundation models.

    For example:

    apiVersion: watsonxaiifm.cpd.ibm.com/v1beta1
kind: Watsonxaiifm
metadata:
name: watsonxaiifm-cr
......
spec:
  ignoreForMaintenance: false
  .......
  custom_foundation_models:
  - location:
      pvc_name: example_model_pvc
    model_id: example_model_70b
    parameters:
    - default: 16
      max: 256
      min: 1
      name: max_num_seqs
    - default: 2048
      max: 8192
      min: 256
      name: max_model_length
    tags:
    - example_model
    - 70b
  - location:
      pvc_name: example_model_pvc_13b
    model_id: example_model_13b
    After registering the custom foundation model in the CR, wait for two minutes to allow the operator to reconcile.