Installing watsonx Orchestrate

Information you need to complete this task

Review the following information before you install watsonx Orchestrate:

Version requirements

All of the components that are associated with an instance of IBM Software Hub must be installed at the same release. For example, if the IBM Software Hub control plane is installed at Version 5.1.3, you must install watsonx Orchestrate at Version 5.1.3.

Environment variables

The commands in this task use environment variables so that you can run the commands exactly as written.

• If you don't have the script that defines the environment variables, see Setting up installation environment variables.
• To use the environment variables from the script, you must source the environment variables before you run the commands in this task. For example, run:

  source ./cpd_vars.sh

Security context constraint

watsonx Orchestrate works with the default Red Hat® OpenShift® Container Platform security context constraint, restricted-v2.

Storage requirements

You must specify storage classes when you install watsonx Orchestrate. The following storage classes are recommended. However, if you don't use these storage classes on your cluster, ensure that you specify a storage class with an equivalent definition.

Storage	Notes	Storage classes
OpenShift Data Foundation	When you install the service, specify file storage and block storage.	File storage: ocs-storagecluster-cephfs Block storage: ocs-storagecluster-ceph-rbd
IBM Fusion Data Foundation	When you install the service, specify file storage and block storage.	File storage: ocs-storagecluster-cephfs Block storage: ocs-storagecluster-ceph-rbd
IBM Fusion Global Data Platform	When you install the service, specify the same storage class for both file storage and block storage.	File storage: Either of the following storage classes: ibm-spectrum-scale-sc ibm-storage-fusion-cp-sc Block storage: Either of the following storage classes: ibm-spectrum-scale-sc ibm-storage-fusion-cp-sc
IBM Storage Scale Container Native	Not supported.	Not applicable.
Portworx	When you install the service, the --storage_vendor=portworx option ensures that the service uses the correct storage classes.	File storage: portworx-shared-gp3 Block storage: portworx-db-gp3-sc
NFS	Not supported.	Not applicable.
Amazon Elastic storage	Not supported.	Not applicable.
NetApp Trident	When you install the service, specify the same storage class for both file storage and block storage.	File storage: ontap-nas Block storage: ontap-nas
Nutanix	Not supported.	Not applicable.

Before you begin

This task assumes that the following prerequisites are met:

Preparing to install

Procedure

Complete the following tasks to install watsonx Orchestrate:

1. Specifying installation options
2. Installing the service
3. Validating the installation
4. Optional: Enabling the models and GPU features
5. What to do next

Specifying installation options

################################################################################
# watsonx Orchestrate parameters
################################################################################
#watson_orchestrate_watsonx_ai_type: false
#watson_orchestrate_syom_models: [] 
#watson_orchestrate_ootb_models: []

Installing the service

To install watsonx Orchestrate:

1. Log the cpd-cli in to the Red Hat OpenShift Container Platform cluster:

   ${CPDM_OC_LOGIN}

   Remember: CPDM_OC_LOGIN is an alias for the cpd-cli manage login-to-ocp command.
2. Important:

   • Do this step only if you are in Version 5.1.0, 5.1.1, or 5.1.2.
   • Before you proceed to deploy the App Connect instance and create App Connect secret, you must create dedicated namespaces for Cloud Pak for Data operators and Cloud Pak for Data instance.
   Create an instance of App Connect for watsonx Orchestrate:

   cpd-cli manage setup-appconnect \
   --appconnect_ns=${PROJECT_IBM_APP_CONNECT} \
   --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
   --release=${VERSION} \
   --components=watsonx_orchestrate \
   --file_storage_class=${STG_CLASS_FILE}

   For more information, see Manage setup App Connect.
3. Run the following command to create the required OLM objects for watsonx Orchestrate in the operators project for the instance:

   cpd-cli manage apply-olm \
   --release=${VERSION} \
   --cpd_operator_ns=${PROJECT_CPD_INST_OPERATORS} \
   --components=watsonx_orchestrate

   Wait for the cpd-cli to return the following message before you proceed to the next step:

   [SUCCESS]... The apply-olm command ran successfully

   If the apply-olm fails, see Troubleshooting the apply-olm command during installation or upgrade.

4. Create the custom resource for watsonx Orchestrate.

   The command that you run depends on the storage on your cluster.

   ---

   Red Hat OpenShift Data Foundation storage

   Run the appropriate command to create the custom resource.

   Default installation (without installation options)

   cpd-cli manage apply-cr \
   --components=watsonx_orchestrate \
   --release=${VERSION} \
   --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
   --block_storage_class=${STG_CLASS_BLOCK} \
   --file_storage_class=${STG_CLASS_FILE} \
   --license_acceptance=true

   Custom installation (with installation options)

   cpd-cli manage apply-cr \
   --components=watsonx_orchestrate \
   --release=${VERSION} \
   --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
   --block_storage_class=${STG_CLASS_BLOCK} \
   --file_storage_class=${STG_CLASS_FILE} \
   --param-file=/tmp/work/install-options.yml \
   --license_acceptance=true

   ---

   IBM Fusion Data Foundation storage

   Run the appropriate command to create the custom resource.

   Default installation (without installation options)

   cpd-cli manage apply-cr \
   --components=watsonx_orchestrate \
   --release=${VERSION} \
   --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
   --block_storage_class=${STG_CLASS_BLOCK} \
   --file_storage_class=${STG_CLASS_FILE} \
   --license_acceptance=true

   Custom installation (with installation options)

   cpd-cli manage apply-cr \
   --components=watsonx_orchestrate \
   --release=${VERSION} \
   --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
   --block_storage_class=${STG_CLASS_BLOCK} \
   --file_storage_class=${STG_CLASS_FILE} \
   --param-file=/tmp/work/install-options.yml \
   --license_acceptance=true

   ---

   Portworx storage

   Run the appropriate command to create the custom resource.

   Default installation (without installation options)

   cpd-cli manage apply-cr \
   --components=watsonx_orchestrate \
   --release=${VERSION} \
   --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
   --storage_vendor=portworx \
   --license_acceptance=true

   Custom installation (with installation options)

   cpd-cli manage apply-cr \
   --components=watsonx_orchestrate \
   --release=${VERSION} \
   --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
   --storage_vendor=portworx \
   --param-file=/tmp/work/install-options.yml \
   --license_acceptance=true

   ---

   NetApp Trident

   Remember: When you use NetApp Trident storage, both ${STG_CLASS_BLOCK} and ${STG_CLASS_FILE} point to the same storage class, typically ontap-nas.

   Run the appropriate command to create the custom resource.

   Default installation (without installation options)

   cpd-cli manage apply-cr \
   --components=watsonx_orchestrate \
   --release=${VERSION} \
   --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
   --block_storage_class=${STG_CLASS_BLOCK} \
   --file_storage_class=${STG_CLASS_FILE} \
   --license_acceptance=true

   Custom installation (with installation options)

   cpd-cli manage apply-cr \
   --components=watsonx_orchestrate \
   --release=${VERSION} \
   --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
   --block_storage_class=${STG_CLASS_BLOCK} \
   --file_storage_class=${STG_CLASS_FILE} \
   --param-file=/tmp/work/install-options.yml \
   --license_acceptance=true

   ---

Validating the installation

[SUCCESS]... The apply-cr command ran successfully

If you want to confirm that the custom resource status is Completed, you can run the cpd-cli manage get-cr-status command:

cpd-cli manage get-cr-status \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--components=watsonx_orchestrate

Optional: Enabling the models and GPU features

Enabling the default models for using AI agent feature in Version 5.1.2

For using the AI agent feature in watsonx Orchestrate, you must enable the default models.

Choose one of the following options:

• You can enable llama-3-1-70b-instruct as default to use AI agent features and other features in watsonx Orchestrate. This model requires a minimum of 4 GPUs.
• You can enable two Granite models: ibm-granite-8b-unified-api-model-v2 and granite-3-8b-instruct. Enabling both the models require a minimum of 2 GPUs. In addition, you must update the models through an API to use the AI agent and other features. For more information, see Updating the Granite models.

To enable llama-3-1-70b-instruct as default, use the following command:

oc patch wo wo \
--namespace=${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch='{
  "spec": {
    "watsonAssistants": {
      "config": {
        "configOverrides": {
          "enabled_components": {
            "store": {
              "ifm": true
            }
          },
          "watsonx_enabled": true,
          "ifm": {
            "model_config": {
              "ootb": {
                "llama-3-1-70b-instruct": {}
              }
            }
          }
        }
      }
    }
  }
}'

To enable ibm-granite-8b-unified-api-model-v2 and granite-3-8b-instruct, use the following command:

oc patch wo wo \
--namespace=${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch='{
  "spec": {
    "watsonAssistants": {
      "config": {
        "configOverrides": {
          "enabled_components": {
            "store": {
              "ifm": true
            }
          },
          "watsonx_enabled": true,
          "ifm": {
            "model_config": {
              "ootb": {
                "granite-3-8b-instruct": {}
              },
              "syom": {
                "ibm-granite-8b-unified-api-model-v2": {}
              }
            }
          }
        }
      }
    }
  }
}'

Updating the Granite models in Version 5.1.2

Note: Before you start to update, you must create a Zen API key using Generating a Zen API key.

To use AI agent and other features, update the Granite models per the following steps:

• Update the Base LLM model to granite-3-8b-instruct.
• Update the model for slot-filling and routing features to granite-8b-unified-api-model-v2.

To update the models, run the following API request as a cURL command:

curl --request PATCH \
  --url https://{root_uri}/orchestrate/{namespace}/instances/{instanceid}/v1/orchestrate/default-agent-settings \
  --header 'Authorization: ZenApiKey {REPLACE_THIS_VALUE}' \
  --header 'Content-Type: application/json' \
  --header 'accept: application/json' \
  --data '{
	"settings": {
    "model_settings": {
        "base_llm": {
            "model_name": "ibm/granite-3-8b-instruct",
            "model_params": {
                "random_seed": 2024,
                "temperature": 0.7,
                "max_new_tokens": 2048,
                "min_new_tokens": 1,
                "decoding_method": "sample"
            },
            "guardrails": {
                "hap": {
                    "input": {
                        "enabled": true,
                        "threshold": 0.5
                    },
                    "output": {
                        "enabled": true,
                        "threshold": 0.5
                    }
                },
                "pii": {
                    "input": {
                        "enabled": true,
                        "threshold": 0.5
                    },
                    "output": {
                        "enabled": true,
                        "threshold": 0.5
                    },
                    "mask": {
                        "remove_entity_value": true
                    }
                }
            },
            "system_prompt": "You are Granite Chat, an AI language model developed by IBM. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior. You always respond to greetings (for example, hi, hello, g'\''day, morning, afternoon, evening, night, what'\''s up, nice to meet you, sup, etc) with \"Hello! I am watsonx Orchestrate, created by IBM. How can I help you today?\". Please do not say anything else and do not start a conversation.\n"
        },
        "embeddings": {
            "model_name": "ibm/slate-125m-english-rtrvr"
        }
    },
    "router_settings": {
        "router_type": "unified",
        "model_name": "ibm/granite-8b-unified-api-model-v2",
        "model_params": {
            "top_k": 50,
            "top_p": 1,
            "random_seed": 2024,
            "temperature": 0.7,
            "max_new_tokens": 200,
            "min_new_tokens": 0,
            "return_options": {
                "input_text": false,
                "token_ranks": false,
                "input_tokens": true,
                "top_n_tokens": 0,
                "token_logprobs": true,
                "generated_tokens": true
            },
            "decoding_method": "sample",
            "repetition_penalty": 1
        },
        "router_config": {
            "continue_journey_on_none": true,
            "confidence_threshold": 0.8,
            "confidence_method": "perplexity"
        }
    },
    "user_settings": {
        "confirm_routing": true,
        "clientside_shortlisting": false
    },
    "slot_filling_settings": {
        "slot_filler_type": "unified",
        "model_name": "ibm/granite-8b-unified-api-model-v2",
        "model_params": {
            "top_k": 50,
            "top_p": 1,
            "random_seed": 2024,
            "temperature": 0.7,
            "max_new_tokens": 200,
            "min_new_tokens": 0,
            "return_options": {
                "input_text": false,
                "token_ranks": false,
                "input_tokens": true,
                "top_n_tokens": 0,
                "token_logprobs": true,
                "generated_tokens": true
            },
            "decoding_method": "sample",
            "repetition_penalty": 1
        }
    }
	}
}'

Where

• <root_uri> is the deployment URL.
• <namespace> is the namespace of your instance. To retrieve the <namespace>, use the Get service instance details API.
• <instanceid> is the unique identifier of your instance. It is the number sequence after orchestrate- in the instance details URL.
• {REPLACE_THIS_VALUE} is the generated Zen API key.

Enabling the GPU feature for Version 5.1.0 and 5.1.1

watsonx Orchestrate uses the watsonx Assistant's GPU feature, which is disabled by default. GPU configuration is a prerequisite to use features such as Skill-based actions and Conversational search. To enable GPU features on watsonx Assistant through watsonx Orchestrate, run the following command.

oc patch wo wo \
--namespace=${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch='{"spec": {"watsonAssistants": {"config": {"configOverrides": {"enabled_components": {"store": {"ifm": true}}, "watsonx_enabled": true}}}}}'

For more information about supported GPUs, see Hardware requirements for watsonx Orchestrate.

What to do next