Installing the stand-alone watsonx.data

Complete the following steps to install IBM® watsonx.data as a stand-alone software.

Before you begin

  • Ensure that Red Hat® OpenShift® version 4.10.0 or later is available.
  • Ensure that the most recent version of podman, httpd-tools, and jq packages are available.
  • Ensure that the credential for production registry is available.

Procedure

  1. Create the environment file.
    1. Set up the work directory. 
      mkdir /root/ibm-lh-manage
      cd /root/ibm-lh-manage/
    2. Create the ibm-lh-manage.env file with the following content. Replace the value for IBM_ENTITLEMENT_KEY. 
      export LH_IMAGE_TAG=latest
export DOCKER_EXE=podman
export UTILS_IMG=icr.io/cpopen/watsonx-data/ibm-lakehouse-manage-utils:$LH_IMAGE_TAG
export IBM_LH_TOOLBOX=icr.io/cpopen/watsonx-data/ibm-lakehouse-toolbox:$LH_IMAGE_TAG
export WORK_DIR=/root/ibm-lh-manage/.ibm-lh-manage-utils
export OCP_URL=<your_Red_Hat_OpenShift_Container_Platform_URL>
export OPENSHIFT_TYPE=self-managed
export IMAGE_ARCH=x86-64
export PROJECT_CPD_OPS=cpd-operator
export PROJECT_CPD_INSTANCE=cpd-instance
export STG_CLASS_BLOCK=<block_storage_class_name_on_a_supported_storage_option>
export STG_CLASS_FILE=<file_storage_class_name_on_a_supported_storage_option>
export PROD_REGISTRY=cp.icr.io
export PROD_USER=cp
export IBM_ENTITLEMENT_KEY=<ibm_entitlement_key>
      Note: For OPENSHIFT_TYPE, specify the type of Red Hat OpenShift Container Platform cluster that you are running. The following are valid values for OPENSHIFT_TYPE:

      aro: Specify aro if you are running Azure Red Hat OpenShift (ARO), the managed OpenShift offering on Microsoft Azure.

      roks: Specify roks if you are running Red Hat OpenShift on IBM Cloud®, the managed OpenShift offering on IBM® Cloud.

      rosa: Specify rosa if you are running Red Hat OpenShift Service on AWS (ROSA), the managed OpenShift offering on Amazon Web Services.

      self-managed (default value): Specify self-managed if you are running self-managed OpenShift on On-premises infrastructure, AWS infrastructure, IBM Cloud infrastructure or Microsoft Azure infrastructure.

      Note: Image tags available for watsonx.data releases: latest, v1.0.0, v1.0.1, v1.0.2, and v1.0.3.
    3. Source the environment file. 
      source ./ibm-lh-manage.env
  2. Pull the watsonx.data ibm-lakehouse-toolbox image. 
    $DOCKER_EXE pull $IBM_LH_TOOLBOX
id=$($DOCKER_EXE create $IBM_LH_TOOLBOX)
$DOCKER_EXE cp $id:/opt - > /tmp/pkg.tar
$DOCKER_EXE rm $id
unset id
  3. Extract the watsonx.data stand-alone pkg.tar file in to the /tmp directory. Verify that the checksum is correct by comparing the checksum in bom.txt and the cksum command output. For example: 
    tar -xf /tmp/pkg.tar -C /tmp
cat /tmp/opt/bom.txt
cksum /tmp/opt/*/*
cp /tmp/opt/standalone/ibm-lakehouse-manage /root/ibm-lh-manage
cp /tmp/opt/standalone/README.txt /root/ibm-lh-manage
  4. Run the following command to initialize the ibm-lh-manage-utils container. 
    /root/ibm-lh-manage/ibm-lakehouse-manage initialize
    Verify that the command generates a directory called .ibm-lh-manage-utils file. For example, 
    ls -a
.  ..  .ibm-lh-manage-utils  ibm-lakehouse-manage
  5. Log in to ocp using one of the following options:

    Option 1: Run the following command to log in to the cluster by providing a username and password:

    ./ibm-lakehouse-manage login-to-ocp \
--user=${OCP_USERNAME} \
--password=${OCP_PASSWORD} \
--server=${OCP_URL}

    Option 2: Run the following command to log in to the cluster by providing a token:

    ./ibm-lakehouse-manage login-to-ocp \
--server=${OCP_URL} \
--token=${OCP_TOKEN}
  6. Add the pull secret to the artifactory that contains watsonx.data images.
    Note:

    For IBM Cloud based installation of watsonx.data, to add the pull secret to the artifactory that contains watsonx.data images, refer to this procedure.

    For AWS ROSA based installation of watsonx.data, to add the pull secret to the artifactory that contains watsonx.data images, follow steps 1 - 5 from this procedure.

    1. Run the following command to create the pull secret:
      oc patch --type=merge --patch='{"spec":{"paused":true}}' machineconfigpool/master
      oc patch --type=merge --patch='{"spec":{"paused":true}}' machineconfigpool/worker
    2. Run the following command:
      /root/ibm-lh-manage/ibm-lakehouse-manage add-icr-cred-to-global-pull-secret --entitled_registry_key=${IBM_ENTITLEMENT_KEY}
      Note: For airgap installation, run the following command to add credentials to the private registry:
      ./ibm-lakehouse-manage add-cred-to-global-pull-secret \
--registry=${PRIVATE_REGISTRY_LOCATION} \
--registry_pull_user=${PRIVATE_REGISTRY_PULL_USER} \
--registry_pull_password=${PRIVATE_REGISTRY_PULL_PASSWORD}
    3. Run the following command:
      oc patch --type=merge --patch='{"spec":{"paused":false}}' machineconfigpool/master
      oc patch --type=merge --patch='{"spec":{"paused":false}}' machineconfigpool/worker
    4. For airgap installation, create the image content resource policy. For more information, see Configuring an image content source policy.
    When the pull secret is created, Red Hat OpenShift propagates it to every node which could take some time to complete. Therefore, wait until the UPDATED column displays True for all the worker nodes in the system config pool before you proceed to the next step. 
    oc get mcp
    For example,
    NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-b7051ee9e86e3e9b84a6513e036fdffb   True     False       False      3              3                   3                     0                      4h57m
worker   rendered-worker-7d2acdbdf2241dbdb7021060eae119a0   True     False       False      3              3                   3                     0                      4h57m
  7. Install watsonx.data.
    1. Run the following command:
      /root/ibm-lh-manage/ibm-lakehouse-manage apply-cluster-components

      When the command completes successfully, you can see the certificate manager pods in ibm-cert-manager namespace and licensing pods in ibm-licensing namespace. 

      oc get po -n ibm-cert-manager
      NAME                                        READY   STATUS    RESTARTS   AGE
cert-manager-cainjector-7bff44479c-sfmdf    1/1     Running   0          3m1s
cert-manager-controller-6b8f694446-zgg9k    1/1     Running   0          3m1s
cert-manager-webhook-7b6c5d8cd6-f5k47       1/1     Running   0          3m1s
ibm-cert-manager-operator-fc87b85b6-n6rbd   1/1     Running   0          3m17s

      oc get po -n ibm-licensing
      ibm-licensing-catalog-bdphw                                       1/1     Running     0          4h23m
ibm-licensing-operator-75b7b44b6d-gmb8s                           1/1     Running     0          4h21m
ibm-licensing-service-instance-58754cb84c-7x285                   1/1     Running     0          4h18m
    2. If you are installing the image tag latest or v1.0.3, run the following command:
      podman exec -it ibm-lakehouse-manage-utils sh 
vi ansible-play/config-vars/release-4.7.3.yml
cpfs:
    csv_version: 4.2.0
    case_version: 4.2.99
    sub_channel: "v4.2"
      Update cpfs and case_version to 4.2.0 in the ansible-play/config-vars/release-4.7.3.yml file.
      cpfs:
    csv_version: 4.2.0
    case_version: 4.2.0
    sub_channel: "v4.2"
    3. Set up the topology. Run the following commands:
      /root/ibm-lh-manage/ibm-lakehouse-manage authorize-instance-topology 
      /root/ibm-lh-manage/ibm-lakehouse-manage setup-instance-topology
      When the command completes successfully, you can see the bedrock pods in cpd-operator namespace.
      oc get po -n ${PROJECT_CPD_OPS}
      NAME                                                              READY   STATUS      RESTARTS   AGE
ibm-common-service-operator-75f559b6f9-qkjdb                      1/1     Running     0          5m23s
ibm-namespace-scope-operator-7c7d94869c-m9sl9                     1/1     Running     0          7m21s
opencloud-operators-8vgkn                                         1/1     Running     0          9m27s
operand-deployment-lifecycle-manager-67ff578fb6-wnvmg             1/1     Running     0          3m24s
    4. Run the following command to create the watsonx.data custom resource:
      /root/ibm-lh-manage/ibm-lakehouse-manage install --license_acceptance=true
    5. Run the following command to verify whether the catalog source is created.
      oc get catalogsource -n ${PROJECT_CPD_OPS}
      NAME DISPLAY TYPE PUBLISHER AGE

cloud-native-postgresql-catalog ibm-cloud-native-postgresql-4.14.0+20230616.111503 grpc IBM 9d

cpd-platform ibm-cp-datacore-3.0.0+20230607.180401.400-65 grpc IBM 7d5h

ibm-watsonx-data-catalog ibm-watsonx-data-1.0.0+20230630.010123.969-linux-amd64 grpc IBM 5h47m

opencloud-operators ibm-cp-common-services-4.0.99 grpc IBM 9d
    6. Verify the csv operator version.
      oc get csv -n ${PROJECT_CPD_OPS}
      NAME                                          DISPLAY                                VERSION   REPLACES                          PHASE
cloud-native-postgresql.v1.18.4               EDB Postgres for Kubernetes            1.18.4    cloud-native-postgresql.v1.18.3   Succeeded
cpd-platform-operator.v4.0.0                  Cloud Pak for Data Platform Operator   4.0.0     cpd-platform-operator.v3.5.0      Succeeded
ibm-cert-manager-operator.v4.0.0              IBM Cert Manager                       4.0.0                                       Succeeded
ibm-common-service-operator.v4.0.0            IBM Cloud Pak foundational services    4.0.0                                       Succeeded
ibm-lakehouse.v1.0.0                          IBM watsonx.data                       0.0.1                                       Succeeded
ibm-namespace-scope-operator.v4.0.0           IBM NamespaceScope Operator            4.0.0                                       Succeeded
ibm-zen-operator.v5.0.0                       IBM Zen Service                        5.0.0                                       Succeeded
operand-deployment-lifecycle-manager.v4.0.0   Operand Deployment Lifecycle Manager   4.0.0                                       Succeeded
  8. Verify the CR status. Run the following command: 
    /root/ibm-lh-manage/ibm-lakehouse-manage get-cr-status
    The result is similar to the following command output:
    # getting status for all installed components...  optionally add --components=<comma separated list of cpd components> for a specific set
# component,CR-kind,CR-name,status,version,creationtimestamp,reconciled-version,operator-info
watsonx_data,WxdAddon,wxdaddon,Completed,1.0.2,2023-08-17T01:27:57Z,1.0.2,IBM watsonx.data operator 1.0.2 build number 1.0.2-1192-20230816-142123-onprem-v1.0.2 cpd_platform,Ibmcpd,ibmcpd-cr,Completed,4.7.2,2023-08-17T00:52:45Z,--,cpdPlatform operator 4.2.0 build 8
zen,ZenService,lite-cr,Completed,5.0.1,2023-08-17T00:55:21Z,5.0.1,zen operator 5.0.1 build 32
  9. Get the WxdAddon CR status, version, and build number. Run the following command: 
    oc get $(oc get Wxdaddon -o name -n ${PROJECT_CPD_INSTANCE}) -o custom-columns='VERSION:status.version,STATUS:status.wxdStatus,BUILD:.status.wxdBuildNumber' -n ${PROJECT_CPD_INSTANCE}
    The command output must be Completed for status.
  10. Get the Cloud Pak for Data login credentials. 
    /root/ibm-lh-manage/ibm-lakehouse-manage get-cpd-instance-details
    The result is similar to the following command output:
    get-cpd-instance-details
CPD Url: <cpd_url>
CPD Username: admin
CPD Password: *****
  11. Access the watsonx.data console.
    1. Log in to Cloud Pak for Data with your credentials.
    2. Go to the Instances page.
    3. Click the overflow menu for the watsonx.data instance and then select Open.
      The watsonx.data home page appears.