Removing foundation models from IBM watsonx.ai

You can remove foundation models or text embedding models from the cluster where IBM watsonx.ai is available. If you deployed AI models that you no longer want to use, remove them from the cluster to free up resources for other models or services.

Before you begin

You can remove foundation models that you previously added. For more information, see Adding foundation models.

To complete this task the first time, you must be the instance administrator who installed the IBM watsonx.ai service.

Procedure

To remove foundation models:

Exclude any models that you want to remove from the list the model IDs specified in install_model_list. Limit the list to include only the models that you want to continue to use.

oc patch watsonxaiifm watsonxaiifm-cr \
--namespace=${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch='{"spec":{"install_model_list":["{model-id-to-keep}","{model-id-to-keep}"]}}'

For example, if you added flan-t5-xxl, granite-13b-instruct-v2, and starcoder-15.5b previously and you now want to remove starcoder-15.5b, use the following command:

oc patch watsonxaiifm watsonxaiifm-cr \
--namespace=${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch='{"spec":{"install_model_list":["google-flan-t5-xxl","ibm-granite-13b-instruct-v2"]}}'

The list of models must include all of the models that you want to keep deployed.

What to do next

If the foundation model that you removed had a custom sharding configuration, remove the sharding details for the foundation model from the custom resource also. For more information, see Changing model sharding configuration in IBM watsonx.ai.