Installing IBM Cloud Pak for Watson AIOps in an air-gapped environment (offline) using a portable device

If your cluster is not connected to the internet, you can deploy an installation of IBM Cloud Pak® for Watson AIOps on your Red Hat® OpenShift® Container Platform cluster by using a portable compute device such as a laptop, or a portable storage device such as a USB device.

In this scenario, your air-gapped (offline) environment has a target registry, and a Red Hat OpenShift cluster on which IBM Cloud Pak for Watson AIOps is to be installed. IBM Cloud Pak for Watson AIOps images are mirrored from the internet to a file system on a portable compute device or a portable storage device. The portable device is then disconnected from the internet and connected in the offline environment, where the images are loaded to the target registry. IBM Cloud Pak for Watson AIOps can then be installed in the offline environment by using the target registry.

Before you begin

Important: The following procedure is based on an Red Hat OpenShift 4.12 environment and includes links for that version. If your environment uses a different supported version of Red Hat OpenShift, ensure that you follow the Red Hat OpenShift documentation for that version.

Installation procedure

  1. Set up the mirroring environment
  2. Download the CASE
  3. Mirror images
  4. Configure storage
  5. Install IBM Cloud Pak for Watson AIOps

1. Set up the mirroring environment

Prerequisites

  • Have access to the following sites and ports:
    • *.icr.io:443 for IBM® Entitled Registry and IBM Cloud Pak for Watson AIOps catalog source
    • *.quay.io:443 for IBM Cloud Pak for Watson AIOps catalog and images. For more information, see Important firewall changes for customers pulling container imagesOpens in a new tab.
    • github.com for CASEs and tools
    • redhat.com for Red Hat OpenShift Container Platform upgrades

1.1 Install and configure Red Hat OpenShift

IBM Cloud Pak for Watson AIOps requires Red Hat OpenShift to be installed and running on your target cluster. You must have administrative access to your Red Hat OpenShift cluster.

For more information about the supported versions of Red Hat OpenShift, see Supported Red Hat OpenShift Container Platform versions. The hardware architecture that you install IBM Cloud Pak for Watson AIOps on must be AMD64.

  1. Install Red Hat OpenShift by using the instructions in the Red Hat OpenShift documentation Opens in a new tab. Information about installing a cluster in a restricted network is given in Mirroring images for a disconnected installation Opens in a new tab.

  2. Install the Red Hat OpenShift command line interface (oc) on your cluster's boot node and run oc login. For more information, see the instructions in Getting started with the Red Hat OpenShift CLIOpens in a new tab.

  3. Ensure that the clocks on your Red Hat OpenShift cluster are synchronized. Each Red Hat OpenShift node in the cluster must have access to an NTP server. Red Hat OpenShift nodes use NTP to synchronize their clocks. IBM Cloud Pak for Watson AIOps runs on Red Hat OpenShift and also has this requirement. Discrepancies between the clocks on the Red Hat OpenShift nodes can cause IBM Cloud Pak for Watson AIOps to experience operational issues. See the Red Hat OpenShift documentation Opens in a new tab for information about how to use a MachineConfig custom resource to configure chrony to connect to your NTP servers.

  4. Optionally configure a custom certificate for IBM Cloud Pak for Watson AIOps to use. You can use either of the following methods:

    • Configure a custom certificate for the Red Hat OpenShift cluster. Follow the instructions in the Red Hat OpenShift documentation Replacing the default ingress certificate. Then, deploy the signing CA certificate into the cluster by following the instructions in the Red Hat OpenShift documentation Replacing the CA Bundle certificate.
    • If you would like to use a custom certificate for IBM Cloud Pak for Watson AIOps only, then after installation is complete follow the instructions in Using a custom certificate.

1.2 Set up a target registry

You must have a local Docker type production-grade registry available in the air-gapped environment to store the IBM Cloud Pak for Watson AIOps images in. The registry must meet the following requirements:

  • supports Docker Manifest V2, Schema 2.
  • supports multi-architecture images.
  • is accessible from the Red Hat OpenShift cluster nodes.
  • allows path separators in the image name.
  • you have the username and password for a user who can read from and write to the registry.
  • must have 197 GB of storage to hold all of the software that is to be transferred to the target registry.

If you do not already have a suitable production-grade registry available, then you must install and configure one. For more information, see About the mirror registry Opens in a new tab in the Red Hat OpenShift documentation.

Important: Do not use the Red Hat OpenShift image registry as your target registry. The Red Hat OpenShift registry does not support multi-architecture images or path separators in the image name.

1.3 Prepare a host

Prepare a portable compute device, or if you are using a portable storage device then a connected compute device, as follows.

You must be able to connect your host to the internet. Your host must be on a Linux® x86_64 or Mac platform with any operating system that the IBM Cloud Pak CLI and the Red Hat OpenShift CLI support. If you are on a Windows® platform, you must run the actions in a Linux® x86_64 VM or from a Windows Subsystem for Linux® (WSL) terminal.

Your portable device and any intermediary devices must have 197 GB of storage to hold all the software that is to be transferred to the local target registry.

Complete the following steps on your host.

  1. Install Docker or Podman. One of these tools is needed for container management.

    To install Podman, see the Podman Installation InstructionsOpens in a new tab.

    To install Docker (for example, on Red Hat® Enterprise Linux®), run the following commands.

    yum check-update
    yum install docker
    

    Note: Docker is not shipped or supported for Red Hat Enterprise Linux (RHEL) 8. The Podman container engine replaced docker as the preferred, maintained, and supported container runtime of choice for Red Hat Enterprise Linux 8 systems. For more information, see Running containers without DockerOpens in a new tab in the Red Hat documentation.

  2. Install the Red Hat OpenShift CLI tool, oc.

    oc is required for Red Hat OpenShift management. For more information, see Getting started with the Red Hat OpenShift CLIOpens in a new tab in the Red Hat OpenShift documentation.

1.4 Install the IBM Catalog Management Plug-in for IBM Cloud Pak®

The IBM Catalog Management Plug-in for IBM Cloud Pak (ibm-pak-plugin) is used for the deployment of IBM Cloud Paks® in a disconnected environment. It simplifies the process for discovering required IBM product images and uses standard tools for registry and cluster access. The ibm-pak-plugin also extends the Red Hat OpenShift CLI (oc) capability to streamline the process of delivering installation images to the IBM Cloud Pak in an air-gapped environment.

  1. Download and install the most recent version of the IBM Catalog Management Plug-in for IBM Cloud Paks for your host operating system from github.com/IBMOpens in a new tab.

  2. Run the following command to extract the files.

    tar -xf oc-ibm_pak-linux-amd64.tar.gz
    
  3. Run the following command to move the file to the /usr/local/bin directory.

    mv oc-ibm_pak-linux-amd64 /usr/local/bin/oc-ibm_pak
    

    Note: If you are installing as a non-root user, you must use sudo.

  4. Confirm that ibm-pak-plugin is installed by running the following command.

    oc ibm-pak --help
    

    Expected result: The ibm-pak-plugin usage is displayed.

1.5 Download documentation and scripts for offline access

Download the following documentation and scripts that you might need to access during your IBM Cloud Pak for Watson AIOps installation, and copy them to your air-gapped environment.

  1. IBM Cloud Pak for Watson AIOps 4.1.2 documentation

    Download the Cloud Pak for Watson AIOps 4.1.2 PDF (this documentation) so that you can access it offline.

  2. IBM Cloud Pak for Watson AIOps 4.1.2 scripts

    • The prerequisite checker script verifies whether your Red Hat OpenShift cluster is correctly set up for a IBM Cloud Pak for Watson AIOps installation. You will need to run this script in step 5.5 Verify cluster readiness. Download this script from github.com/IBMOpens in a new tab.

    • An uninstall script can be downloaded from github.com/IBMOpens in a new tab.

    • (Optional) The status checker script can be used in step 5.7 Create the custom resource to give information about the status of your deployment. The use of this script is optional, as status can be found directly from the ibm-aiops-orchestrator custom resource. This script can be downloaded from github.com/IBMOpens in a new tab.

  3. Red Hat OpenShift documentation

    The Red Hat OpenShift documentation can be downloaded for offline access from Red HatOpens in a new tab. The CLI tools, Images, Operators, and Networking sections are referenced by this documentation.

2. Download the CASE

Set environment variables on the portable device, and connect it to the internet so that you can download the IBM Cloud Pak for Watson AIOps CASE files.

Note: Save a copy of your environment variable values to a file that you can use as a reference when you are completing your air-gapped installation tasks.

  1. Create the following environment variables.

    export CASE_NAME=ibm-cp-waiops
    export CASE_VERSION=1.7.2
    export CASE_INVENTORY_SETUP=cpwaiopsSetup
    export TARGET_REGISTRY_HOST=<IP_or_FQDN_of_target_registry>
    export TARGET_REGISTRY_PORT=<port_number_of_target_registry>
    export TARGET_REGISTRY=$TARGET_REGISTRY_HOST:$TARGET_REGISTRY_PORT
    export TARGET_REGISTRY_USER=<username>
    export TARGET_REGISTRY_PASSWORD=<password>
    

    The target registry is the registry where the IBM Cloud Pak for Watson AIOps images are mirrored to, and accessed from by the Red Hat OpenShift cluster, as setup in 1.2 Set up a target registry

    If your portable device must connect to the internet through a proxy, then also set the following environment variables.

    export https_proxy=http://proxy-server-hostname:port
    export http_proxy=http://proxy-server-hostname:port
    
  2. Connect your portable device to the internet and disconnect it from the air-gapped environment.

  3. Download the IBM Cloud Pak for Watson AIOps installer and image inventory to your portable device.

    oc ibm-pak get ${CASE_NAME} --version ${CASE_VERSION}
    

    The CASE is downloaded to ~/.ibm-pak/data/cases/$CASE_NAME/$CASE_VERSION. The logs files are available at ~/.ibm-pak/logs/oc-ibm_pak.log.

    Note: If you do not specify the CASE version, then the latest CASE is downloaded. The root directory that is used by the ibm-pak-plugin is ~/.ibm-pak. If required, the root directory can be configured by setting the IBMPAK_HOME environment variable.

3. Mirror images

Complete the following steps to mirror the IBM Cloud Pak for Watson AIOps images from the internet to the file system on your portable device, and then from the file system to the target registry in the air-gapped environment.

3.1. Generate mirror manifests

Run the following command to generate mirror manifests to be used when mirroring the images to the target registry.

oc ibm-pak generate mirror-manifests ${CASE_NAME} file://cpfs --version ${CASE_VERSION} --final-registry ${TARGET_REGISTRY}/cpfs
Table 1. Parameter description
Argument Description
file://cpfs Specifies the path extension where images are mirrored to. Images are mirrored to $IMAGE_PATH/cpfs when the oc image mirror command is run with images-mapping-to-filesystem.txt. For more information, see Mirror the images to the file system.
$TARGET_REGISTRY/cpfs This argument generates an image-mapping file that is used by the oc image mirror commands to mirror images to the TARGET_REGISTRY at namespace cpfs. For example, if the CASE you are installing needs the image, quay.io/opencloudio/ibm-events-kafka-2.6.0@sha256:10d422dddd29ff19c87066fc6510eee05f5fa4ff608b87a06e898b3b6a3a13c7, then its final URL in your target registry will be $TARGET_REGISTRY/cpfs/opencloudio/ibm-events-kafka-2.6.0. Note the new namespace of cpfs in the URL. The namespace path can be multi level if your target registry supports it.

The command generates the following files at ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION:

  • images-mapping-to-filesystem.txt
  • images-mapping-from-filesystem.txt
  • image-content-source-policy.yaml

Example ~/.ibm-pak directory structure

The ~/.ibm-pak directory structure is built over time as you save CASEs and mirror. The following tree shows an example of the ~/.ibm-pak directory structure for CASE version 1.7.2:

[root@bastion-cp4waiops-testx-cluster ~]# tree .ibm-pak/
.ibm-pak/
├── config
│   └── config.yaml
├── data
│   ├── cases
│   │   └── ibm-cp-waiops
│   │       └── 1.7.2
│   │           ├── caseDependencyMapping.csv
│   │           ├── charts
│   │           ├── component-set-config.yaml
│   │           ├── ibm-aiops-analytics-2.5.0-airgap-metadata.yaml
│   │           ├── ibm-aiops-analytics-2.5.0-charts.csv
│   │           ├── ...
│   │           └── resourceIndexes
│   │               ├── ibm-aiops-analytics-resourcesIndex.yaml
│   │               ├── ibm-aiopsedge-bundle-resourcesIndex.yaml
│   │               ├── ...
│   └── mirror
│       └── ibm-cp-waiops
│           └── 1.7.2
│               ├── catalog-sources.yaml
│               ├── image-content-source-policy.yaml
│               └── images-mapping.txt
└── logs
    └── oc-ibm_pak.log

3.2. Authenticate with the IBM Entitled Registry

Log in to the IBM Entitled Registry to generate an authentication file containing the IBM Entitled Registry credentials, and then create an environment variable that has the location of the authentication file. This file is used later to enable the oc image mirror command to pull the images from the IBM Entitled Registry.

  1. Get the authentication credentials for the IBM Entitled Registry.

    1. To obtain the entitlement key that is assigned to your IBMid, log in to MyIBM Container Software Library Opens in a new tab with the IBMid and password details that are associated with the entitled software.

    2. In the Entitlement keys section, select Copy key to copy the entitlement key.

  2. Run the following command to create an environment variable that contains your entitlement key.

    export ENTITLED_REGISTRY_PASSWORD=<key>
    

    Where <key> is the entitlement key that you copied in the previous step.

  3. Store the authentication credentials for the IBM Entitled Registry.

    If you are using Podman, run the following commands:

    podman login cp.icr.io -u cp -p ${ENTITLED_REGISTRY_PASSWORD}
    export REGISTRY_AUTH_FILE=${XDG_RUNTIME_DIR}/containers/auth.json
    unset ENTITLED_REGISTRY_PASSWORD
    

    Note: The authentication file is usually at ${XDG_RUNTIME_DIR}/containers/auth.json. For more information, see the Options section in the Podman documentationOpens in a new tab.

    If you are using Docker, run the following commands:

    docker login cp.icr.io -u cp -p ${ENTITLED_REGISTRY_PASSWORD}
    export REGISTRY_AUTH_FILE=$HOME/.docker/config.json
    unset ENTITLED_REGISTRY_PASSWORD
    

    Note: The authentication file is usually at $HOME/.docker/config.json. For more information, see the Docker documentationOpens in a new tab.

3.3. Mirror the images to the file system

Complete these steps to mirror the images from the internet to a file system on your portable device.

  1. Create an environment variable to store the location of the file system where the images are to be stored.

    export IMAGE_PATH=<image-path>
    

    Where <image-path> is the directory where you want the images to be stored.

  2. Run the following command to mirror the images from the IBM Entitled Registry to the file system.

    nohup oc image mirror \
    -f ~/.ibm-pak/data/mirror/${CASE_NAME}/${CASE_VERSION}/images-mapping-to-filesystem.txt \
    -a ${REGISTRY_AUTH_FILE} \
    --filter-by-os '.*' \
    --insecure \
    --skip-multiple-scopes \
    --dir "${IMAGE_PATH}" \
    --max-per-registry=1 > my-mirror-progress.txt 2>&1 &
    

    The UNIX® command nohup is used to ensure that the mirroring process continues even if there is a loss of network connection, and redirection of output to a file provides improved monitoring and error visibility.

    Run the following command if you want to see the progress of the mirroring:

    tail -f my-mirror-progress.txt
    

    Note: If an error occurs during mirroring, the mirror command can be rerun.

3.4 Setup the file system in the air-gapped environment

  1. Copy files to the air-gapped environment (portable storage device only)

    If you are using a portable storage device, you must copy the files from the portable storage device to a local compute device in the air-gapped environment that has access to the target registry. If you are using a portable compute device, then these items are already present, and you can proceed to the next step.

    Copy the following items to your local compute device:

    • the file system located at $IMAGE_PATH, which you specified earlier
    • ~/.ibm-pak directory
  2. Disconnect the device that has your file system, (the portable compute device or the local compute device) from the internet and connect it to the air-gapped environment.

  3. Ensure that environment variables are set on the device in the air-gapped environment that has access to the target registry.

    If you are using a portable storage device, then set the following environment variables on your local compute device within the air-gapped environment.

    If you are using a portable compute device that you have restarted since mirroring the images, then your environment variables will have been lost and you will need to set the following environment variables on your portable compute device again.

    export CASE_NAME=ibm-cp-waiops
    export CASE_VERSION=1.7.2
    export CASE_INVENTORY_SETUP=cpwaiopsSetup
    export TARGET_REGISTRY_HOST=<IP_or_FQDN_of_target_registry>
    export TARGET_REGISTRY_PORT=<port_number_of_target_registry>
    export TARGET_REGISTRY=${TARGET_REGISTRY_HOST}:${TARGET_REGISTRY_PORT}
    export TARGET_REGISTRY_USER=<username>
    export TARGET_REGISTRY_PASSWORD=<password>
    export IMAGE_PATH=<image_path>
    

3.5 Authenticate with the target registry

Authenticate with the target registry in the air-gapped environment that you are mirroring the images into.

If you are using Podman, run the following command:

podman login ${TARGET_REGISTRY} -u ${TARGET_REGISTRY_USER} -p ${TARGET_REGISTRY_PASSWORD}
export REGISTRY_AUTH_FILE=${XDG_RUNTIME_DIR}/containers/auth.json

Note: The authentication file is usually at ${XDG_RUNTIME_DIR}/containers/auth.json. For more information, see the Options section in the Podman documentationOpens in a new tab.

If you are using Docker, run the following command:

docker login ${TARGET_REGISTRY} -u ${TARGET_REGISTRY_USER} -p ${TARGET_REGISTRY_PASSWORD}
export REGISTRY_AUTH_FILE=$HOME/.docker/config.json

Note: The authentication file is usually at $HOME/.docker/config.json. For more information, see the Docker documentationOpens in a new tab.

3.6 Mirror the images to the target registry from the file system

Complete the steps in this section on the device that has your file system (the portable compute device or the local compute device) to copy the images from the file system to the $TARGET_REGISTRY. Your device with the file system must be connected to both the target registry and the Red Hat OpenShift cluster.

Run the following command to copy the images referenced in the images-mapping-from-filesystem.txt from the $IMAGE_PATH file system to the final target registry.

nohup oc image mirror \
-f ~/.ibm-pak/data/mirror/${CASE_NAME}/${CASE_VERSION}/images-mapping-from-filesystem.txt \
--from-dir "${IMAGE_PATH}" \
-a ${REGISTRY_AUTH_FILE} \
--filter-by-os '.*' \
--insecure  \
--skip-multiple-scopes \
--max-per-registry=1 > my-mirror-progress2.txt 2>&1 &

The UNIX command nohup is used to ensure that the mirroring process continues even if there is a loss of network connection, and redirection of output to a file provides improved monitoring and error visibility.

Run the following command if you want to see the progress of the mirroring:

tail -f my-mirror-progress2.txt

Note: If an error occurs during mirroring, the mirror command can be rerun.

3.7 Configure the cluster

  1. Log in to your Red Hat OpenShift cluster.

    You can identify your specific oc login command by clicking the user menu in the upper left of the Red Hat OpenShift console, and then clicking Copy Login Command.

    Example:

    oc login <server> -u <cluster username> -p <cluster pass>
    
  2. Update the global image pull secret for your Red Hat OpenShift cluster.

    Follow the steps in the Red Hat OpenShift documentation topic Updating the global cluster pull secret Opens in a new tab.

    These steps enable your cluster to have authentication credentials in place to pull images from your TARGET_REGISTRY as specified in the image-content-source-policy.yaml, which you will apply to your cluster in the next step.

  3. Create the ImageContentSourcePolicy.

    Run the following command to create an ImageContentSourcePolicy (ICSP).

    oc apply -f  ~/.ibm-pak/data/mirror/${CASE_NAME}/${CASE_VERSION}/image-content-source-policy.yaml
    
  4. Verify that the ImageContentSourcePolicy resource is created.

    oc get imagecontentsourcepolicy
    

    Example output, showing a newly created ibm-cp-waiops ICSP.

    oc get imagecontentsourcepolicy
    NAME                      AGE
    ibm-cp-waiops             95s
    redhat-operator-index-0   5d18h
    
  5. Verify your cluster node status.

    oc get MachineConfigPool -w
    

    Important: After the ImageContentsourcePolicy and global image pull secret are applied, the configuration of your nodes will be updated sequentially. Wait until all the MachineConfigPools are updated before proceeding to the next step.

  6. (Optional) If you use an insecure registry, you must add the target registry to the cluster's insecureRegistries list.

    oc patch image.config.openshift.io/cluster --type=merge \
    -p '{"spec":{"registrySources":{"insecureRegistries":["'${TARGET_REGISTRY}'"]}}}'
    

4. Configure storage

The storage configuration must satisfy your sizing requirements. Two storage classes are needed for installing IBM Cloud Pak for Watson AIOps. For more information, see Storage.

5. Install IBM Cloud Pak for Watson AIOps

Now that the images are mirrored to your air-gapped environment, you can deploy IBM Cloud Pak for Watson AIOps to that environment. To install IBM Cloud Pak for Watson AIOps, complete the following steps.

5.1 Create the catalog source

Now that the images are mirrored to your air-gapped environment, you can deploy IBM Cloud Pak for Watson AIOps to that environment

  1. Create and configure the catalog source.

    oc ibm-pak launch \
    ${CASE_NAME} \
    --version ${CASE_VERSION} \
    --action install-catalog \
    --inventory ${CASE_INVENTORY_SETUP} \
    --namespace openshift-marketplace \
    --args "--registry ${TARGET_REGISTRY}/cpfs --recursive \
    --inputDir ~/.ibm-pak/data/cases/${CASE_NAME}/${CASE_VERSION}"
    
  2. Verify that the CatalogSource is installed.

    oc get pods -n openshift-marketplace
    oc get catalogsource -n openshift-marketplace
    

5.2 Create environment variables

Create and then source a shell script named waiops_var.sh that defines the environment variables which will be used to provide installation parameters for your deployment. Use the following codeblock as a template, replacing the brackets < ... > with values for your environment.

You can use the following table to find the values to set for your storage environment variables.

Table. Storage provider classes
Storage provider RWX-storage-class-name RWO-storage-class-name
IBM Cloud® ibmc-file-gold-gid ibmc-block-gold
Red Hat® OpenShift® Data Foundation ocs-storagecluster-cephfs ocs-storagecluster-ceph-rbd
IBM Storage Fusion ibm-spectrum-scale-sc ibm-spectrum-scale-sc
IBM Spectrum Scale Container Native ibm-spectrum-scale-sc ibm-spectrum-scale-sc
Portworx portworx-fs portworx-aiops
#========================================================================================================
# Cloud Pak for Watson AIOps installation variables
#========================================================================================================

export CP4WAIOPS_NAME=ibm-cp-watson-aiops
export CP4WAIOPS_SIZE=large # Set to small if you only require a starter non-production deployment.
export SECURE_TUNNEL=false # Set to `true` to install Secure Tunnel, otherwise set to `false`.
export PROJECT_CP4WAIOPS=cp4waiops
export ACCEPT_LICENSE=false # Set to `true` to agree to the license terms, otherwise install will fail.
export CATALOG_SRC_CP4WAIOPS=ibm-cp-waiops-catalog

# -------------------------------------------------------------------------------------------------------
# Storage
# -------------------------------------------------------------------------------------------------------
export STG_CLASS=<RWX-storage-class-name>
export STG_CLASS_BLOCK=<RWO-storage-class-name>

# -------------------------------------------------------------------------------------------------------
# Your customer details
# -------------------------------------------------------------------------------------------------------
export CUSTOMER_NAME=<your company name>
export CUSTOMER_ICN=<your IBM Customer Number>
export CUSTOMER_ENVIRONMENT=<Set to `trial`, `poc`, or `production`>

# -------------------------------------------------------------------------------------------------------
# `OwnNamespace` installation mode:  leave INSTALL_MODE_NAMESPACE as it is.
# `AllNamespaces` installation mode: change to export INSTALL_MODE_NAMESPACE=openshift-operators
# -------------------------------------------------------------------------------------------------------
export INSTALL_MODE_NAMESPACE=${PROJECT_CP4WAIOPS}

# -------------------------------------------------------------------------------------------------------
# Topology resource group terminology
# Specify `application` or `service` as the terminology to be used for collections of topology resource
# groups. The default is `application`.
# -------------------------------------------------------------------------------------------------------
export TOPOLOGY_TERMINOLOGY=application

If you need help with deciding on the values to set for these environment variables, see the following topics.

Note: You can set a different value for $PROJECT_CP4WAIOPS and $CP4WAIOPS_NAME if you want. However, you must not use the default, kube-system, kube-public, openshift-node, openshift-infra, or openshift projects (namespaces) for $PROJECT_CP4WAIOPS. This is because IBM Cloud Pak for Watson AIOps uses Security Context Constraints (SCC), and SCCs cannot be assigned to pods created in one of the default Red Hat OpenShift projects (namespaces).

Run the following command to source your script and set the environment variables:

. ./waiops_var.sh

5.3 Create a custom project (namespace)

Run the following command to create a project (namespace) to deploy IBM Cloud Pak for Watson AIOps into.

oc create namespace ${PROJECT_CP4WAIOPS}

5.4. Configure usage data collection

To help the development of IBM Cloud Pak for Watson AIOps, daily aggregated usage data is collected to analyse how IBM Cloud Pak for Watson AIOps is used. The collection of usage data is enabled by default, but can be disabled. Usage data is collected by the cp4waiops-metricsprocessor pod, and stored in the cp4waiops-metricsprocessor pod's logs. This usage data is sent to IBM when MustGather output is sent to IBM, as the MustGather includes the output from the cp4waiops-metricsprocessor pod's logs. The usage data is then sent to and stored in IBM controlled GDPR-compliant systems. The usage data that is collected is all numeric, and does not include email addresses, passwords, or specific details. Only the following data is collected:

  • Current number of applications
  • Current number of alerts (all severities aggregated)
  • Current number of incidents (all priorities aggregated)
  • Current number of policies (includes predefined and user created)
  • Current number of runbooks run since installation
  • Current number of connectors of each type (For example ServiceNow, Instana, Falcon Logscale)

This usage data is sent to IBM only if the MustGather tool is run and sent to IBM. The MustGather includes the output from the cp4waiops-metricsprocessor pod's logs.

Configuring the collection of usage data

If you do not want to disable the collection of usage data, run the following command to configure the usage data with your customer details.

oc create secret generic aiops-metrics-processor -n ${PROJECT_CP4WAIOPS} --from-literal=customerName=${CUSTOMER_NAME} --from-literal=customerICN=${CUSTOMER_ICN} --from-literal=environment=${CUSTOMER_ENVIRONMENT}

Important: Usage data without your customer details is still collected even if you do not create this secret. If you do not want any usage data collected, then you must run the command given in Disabling the collection of usage data.

Disabling the collection of usage data

If you want to disable the collection of usage data, run the following command.

oc create secret generic aiops-metrics-processor -n ${PROJECT_CP4WAIOPS} --from-literal=customerName=${CUSTOMER_NAME} --from-literal=customerICN=${CUSTOMER_ICN} --from-literal=environment=${CUSTOMER_ENVIRONMENT} --from-literal=enableCollection=false 

Note: You can update your usage data collection preferences after installation. For more information, see Updating usage data collection preferences.

5.5 Verify cluster readiness

Run the following command to switch to your IBM Cloud Pak for Watson AIOps project.

oc project ${PROJECT_CP4WAIOPS}

Run the prerequisite checker script that you downloaded in step 1.5 Download documentation and scripts for offline access to verify whether your Red Hat OpenShift cluster is correctly set up for an IBM Cloud Pak for Watson AIOps installation.

Example output:

# ./prereq.sh 
Using project "cp4waiops" on server "https://myserver.mycluster.mydomain:6443".

Starting IBM Cloud Pak for Watson AIOps AI Manager prerequisite checker v4.1...

[INFO] =================================Openshift Container Platform Version Check=================================
[INFO] Checking OCP Version. Compatible Versions of OCP are v4.10.46+ and v4.12.x
[INFO] OCP version 4.12.18 is compaitble
[INFO] =================================Openshift Container Platform Version Check=================================

[INFO] =================================Entitlement Pull Secret=================================
[INFO] Checking whether the Entitlement secret or Global pull secret is configured correctly.
[INFO] Checking if the job 'cp4waiops-entitlement-key-test-job' already exists.
[INFO] The job with name 'cp4waiops-entitlement-key-test-job' was not found, so moving ahead and creating it.
[INFO] Creating the job 'cp4waiops-entitlement-key-test-job'
job.batch/cp4waiops-entitlement-key-test-job created
[INFO] Verifying if the job 'cp4waiops-entitlement-key-test-job' completed successfully..
[INFO] SUCCESS! Entitlement secret is configured correctly.
job.batch "cp4waiops-entitlement-key-test-job" deleted
[INFO] =================================Entitlement Pull Secret=================================

[INFO] =================================Storage Provider=================================
[INFO] Checking storage providers
[INFO] No IBM Storage Fusion Found... Skipping configuration check.

[INFO] No Portworx StorageClusters found with "Running" or "Online" status. Skipping configuration check for Portworx.
[INFO] Openshift Data Foundation found.
[INFO] No IBM Cloud Storage found... Skipping configuration check for IBM Cloud Storage Check.

Checking Openshift Data Foundation Configuration...
Verifying if Red Hat Openshift Data Foundation pods are in "Running" or "Completed" status
[INFO] Pods in openshift-storage project are "Running" or "Completed"
[INFO] ocs-storagecluster-ceph-rbd exists.
[INFO] ocs-storagecluster-cephfs exists.
[INFO] No warnings or failures found when checking for Storage Providers.
[INFO] =================================Storage Provider=================================

[INFO] =================================Small or Large Profile Install Resources=================================
[INFO] Checking for cluster resources

[INFO] ==================================Resource Summary=====================================================
[INFO]                                       Nodes   |      vCPU      |     Memory(GB)
[INFO] Small profile(available/required)  [  14 / 3 ]   [  255 / 62 ]       [  479 / 140 ]
[INFO] Large profile(available/required)  [  14 / 10 ]   [  255 / 162 ]       [  479 / 360 ]
[INFO] ==================================Resource Summary=====================================================
[INFO] Cluster currently has resources available to create a large profile of Cloud Pak for Watson AIOps AI Manager
[INFO] =================================Small or Large Profile Install Resources=================================


[INFO] =================================Prerequisite Checker Tool Summary=================================
      [  PASS  ] Openshift Container Platform Version Check
      [  PASS  ] Entitlement Pull Secret
      [  PASS  ] Storage Provider
      [  PASS  ] Small or Large Profile Install Resources
[INFO] =================================Prerequisite Checker Tool Summary=================================

5.6 Install the operator

For more information about installing operators on a cluster, see the Red Hat OpenShift documentation Adding Operators Opens in a new tab.

For more information about the operators which are installed with IBM Cloud Pak for Watson AIOps, see Operator Details.

  1. Create an OperatorGroup.

    Important: Skip this step if you are installing using the 'All Namespaces' installation mode. Check that you set INSTALL_MODE_NAMESPACE correctly in step 5.2, and proceed to the next step, Install the IBM Cloud Pak for Watson AIOps operator.

    If you are installing using the 'OwnNamespace' installation mode, then you must create an operator group in your custom project (namespace), or the IBM Cloud Pak for Watson AIOps operator will not install. There might be an operator group for managing a namespace for given APIs. If there is an operator group for the namespace, do not create a second one.

    Create the Operator group by running the following command:

    cat << EOF | oc apply -f -
    apiVersion: operators.coreos.com/v1
    kind: OperatorGroup
    metadata:
      name: cp4waiops-operator-group
      namespace: ${PROJECT_CP4WAIOPS}
    spec:
      targetNamespaces:
        - "${PROJECT_CP4WAIOPS}"
    EOF
    
  2. Install the IBM Cloud Pak for Watson AIOps operator.

    Run the following command.

    cat << EOF | oc apply -f -
    apiVersion: operators.coreos.com/v1alpha1
    kind: Subscription
    metadata:
      name: ibm-aiops-orchestrator
      namespace: $INSTALL_MODE_NAMESPACE
    spec:
      channel: v4.1
      installPlanApproval: Automatic
      name: ibm-aiops-orchestrator
      source: ${CATALOG_SRC_CP4WAIOPS}
      sourceNamespace: openshift-marketplace
    EOF
    

    Warning: installPlanApproval must not be changed to Manual. Manual approval, which requires the manual review and approval of the generated InstallPlans, is not supported. Incorrect timing or ordering of manual approvals of InstallPlans can result in a failed installation.

  3. After a few minutes, the IBM Cloud Pak for Watson AIOps operator is installed. Verify that the all of the components have a state of Succeeded by running the following command:

    oc get csv -n ${INSTALL_MODE_NAMESPACE} | egrep "ibm-aiops-orchestrator|ibm-automation|ibm-common-service"
    

    Example output:

    $ oc get csv -n ${INSTALL_MODE_NAMESPACE} | egrep "ibm-aiops-orchestrator|ibm-automation|ibm-common-service"
    ibm-aiops-orchestrator.v4.1.2         IBM Cloud Pak for Watson AIOps AI Manager   4.1.2                                      Succeeded
    ibm-automation-elastic.v1.3.13        IBM Elastic                                 1.3.13   ibm-automation-elastic.v1.3.12    Succeeded
    ibm-automation-flink.v1.3.13          IBM Automation Foundation Flink             1.3.13   ibm-automation-flink.v1.3.12      Succeeded
    ibm-common-service-operator.v3.23.3   IBM Cloud Pak foundational services         3.23.3                                     Succeeded
    

    Important: A dependency may not yet be present when this command is run, and cause IBM Elastic to be Failed. This dependency is resolved when the custom resource is created and rollout continues. No user intervention or delay is required.

5.7 Create the custom resource

Create an instance of the IBM Cloud Pak for Watson AIOps custom resource. A maximum of one IBM Cloud Pak for Watson AIOps custom resource is allowed per cluster.

  1. Use the following YAML to create an instance of the IBM Cloud Pak for Watson AIOps custom resource.

    cat << EOF | oc apply -f -
    apiVersion: orchestrator.aiops.ibm.com/v1alpha1
    kind: Installation
    metadata:
      name: ${CP4WAIOPS_NAME}
      namespace: ${PROJECT_CP4WAIOPS}
    spec:
      size: ${CP4WAIOPS_SIZE}
      storageClass: ${STG_CLASS}
      storageClassLargeBlock: ${STG_CLASS_BLOCK}
      imagePullSecret:
      topologyModel: ${TOPOLOGY_TERMINOLOGY}
      license:
        accept: ${ACCEPT_LICENSE}
      pakModules:
      - name: aiopsFoundation
        enabled: true
      - name: applicationManager
        enabled: true
      - name: aiManager
        enabled: true
      - name: connection
        enabled: ${SECURE_TUNNEL}
    EOF
    

    Warning: The pakModules aiopsFoundation, applicationManager, and aiManager must be enabled as in the following YAML. Do not change these values to false.

  2. Verify your installation.

    Run the following command to check that the PHASE of your installation is Updating.

    oc get installations.orchestrator.aiops.ibm.com -n ${PROJECT_CP4WAIOPS}
    

    Example output:

    NAME                  PHASE     LICENSE    STORAGECLASS   STORAGECLASSLARGEBLOCK   AGE
    ibm-cp-watson-aiops   Updating  Accepted   rook-cephfs    rook-ceph-block          3m
    

    It takes around 60-90 minutes for the installation to complete (subject to the speed with which images can be pulled). When installation is complete and successful, the PHASE of your installation changes to Running. If your installation phase does not change to Running, then use the following command to find out which components are not ready:

    oc get installation.orchestrator.aiops.ibm.com -o yaml | grep 'Not Ready'
    

    Example output:

    lifecycleservice: Not Ready
    zenservice: Not Ready
    

    To see details about why a component is Not Ready run the following command, where <component> is the component that is not ready, for example zenservice.

    oc get <component> -o yaml
    

    (Optional) If you downloaded the status checker script earlier in step 1.5 Download documentation and scripts for offline access, then you can also run this script to see information about the status of your deployment.

If the installation fails, or is not complete and is not progressing, then see Troubleshooting installation and upgrade and Known Issues to help you identify any installation problems.

5.8 (Optional) Create an EgressNetworkPolicy

There is no egress firewall policy defined when you install IBM Cloud Pak for Watson AIOps, so outgoing traffic from workload pods to the internal and external network is unrestricted.

If you require a more secure environment, then use the following steps.

  1. Create an EgressNetworkPolicy on your Red Hat OpenShift cluster to limit egress from the IBM Cloud Pak for Watson AIOps project (namespace).

    For more information about creating an EgressNetworkPolicy, see Configuring an egress firewall for a projectOpens in a new tab.

    Note: There must be only one EgressNetworkPolicy per project/namespace.

  2. Configure exceptions to the EgressNetworkPolicy.

    Edit your EgressNetworkPolicy to add exceptions for the following IBM Cloud Pak for Watson AIOps components that have egress dependencies, otherwise these IBM Cloud Pak for Watson AIOps components fail when they attempt egress.

    1. Allow egress to any external services, such as the following connections:
      • Kubernetes
      • GitHub
      • Microsoft® Teams
      • ServiceNow
      • Slack
      • VMware® vCenter
    2. Configure your EgressNetworkPolicy to allow traffic for your GitHub, Kubernetes, ServiceNow, and VMware vCenter connections.

      Edit your EgressNetworkPolicy to allow or deny egress, as in the following example:

      kind: EgressNetworkPolicy
      metadata:
        name: default
      spec:
        egress:
        - type: Allow
          to:
            cidrSelector: <1.2.3.0/24>
        - type: Allow
          to:
            dnsName: <www.github.com>
        - type: Allow
          to:
            dnsName: <www.developer.kubernetes.com>
        - type: Allow
          to:
            dnsName: <www.developer.servicenow.com>
        - type: Allow
          to:
            dnsName: <www.developer.vcenter.com>
        - type: Deny
          to:
            cidrSelector: <0.0.0.0/0>
      

      Substitute values for `dnsName` and `cidrSelector` that are the DNS names and addresses of your GitHub, Kubernetes, ServiceNow, or VMware vCenter sources.

5.9 Access the Automation console

After you successfully install IBM Cloud Pak for Watson AIOps, get the URL for accessing the Automation console.

  1. Use the following command to get the URL to access the Automation console:

    oc get route -n ${PROJECT_CP4WAIOPS} cpd -o jsonpath='{.spec.host}'
    

    The following output is a sample output:

    cpd-cp4waiops.apps.mycluster.mydomain
    

    Based on the sample output, your console URL would be https://cpd-cp4waiops.apps.mycluster.mydomain

  2. Enter the URL in your browser to open the Automation console. Log in with your username and password.

Find the IBM Cloud Pak for Watson AIOps console username and password

The default username to access the Automation console is admin. You can check the default username and their password with the following commands.

Note: This information is for the IBM provided credentials (admin only) authentication type.

  1. Find the default username.

    oc -n ibm-common-services get secret platform-auth-idp-credentials -o jsonpath='{.data.admin_username}' | base64 -d && echo
    
  2. Get the password for the admin username.

    oc -n ibm-common-services get secret platform-auth-idp-credentials -o jsonpath='{.data.admin_password}' | base64 -d
    

    The following extract shows a sample output:

    EwK9dj9fwPZHyHTyu9TyIgh9klZSzVsA
    

    Based on the sample output, your password would be EwK9dj9fwPZHyHTyu9TyIgh9klZSzVsA.

Important: You can change this default password at any time. For more information, see Changing the cluster administrator password.

What to do next

  • Define integrations and applications with Defining.
  • If you have an existing on-premises IBM Tivoli Netcool/OMNIbus deployment, then you can connect IBM Cloud Pak for Watson AIOps to it with the Netcool connector. For more information, see Creating IBM Tivoli Netcool/OMNIbus connections.
  • If you have an existing on-premises IBM Tivoli Netcool/Impact deployment, then you can connect IBM Cloud Pak for Watson AIOps to it with the IBM Tivoli Netcool/Impact connector. For more information, see Creating IBM Tivoli Netcool/Impact connections.
  • Familiarize yourself with backup and restore procedures. It is recommended that you take regular backups of your IBM Cloud Pak for Watson AIOps deployment. For more information, see Backup and restore.