Preparing to install IBM Cloud Pak for Watson AIOps Infrastructure Automation in an air-gapped environment (offline) using a portable compute device

Prerequisites

You must satisfy the following prerequisites:

• Red Hat® OpenShift® Container Platform requires you to have cluster admin access to run the configure-cluster-airgap and install-operator commands.

• Access to the following sites and ports:

  • *.icr.io:443 for IBM Entitled Registry and Infrastructure Automation catalog source
  • *.quay.io:443 for Infrastructure Automation catalog and images. For more information, see Important firewall changes for customers pulling container images.
  • github.com for CASEs and tools
  • redhat.com for Red Hat OpenShift Container Platform upgrades

1.1 Install and configure Red Hat OpenShift

Infrastructure Automation requires Red Hat OpenShift to be installed and running on your target cluster. You must have administrative access to your Red Hat OpenShift cluster.

For more information about the supported versions of Red Hat OpenShift, see Supported Red Hat OpenShift Container Platform versions.

1. Install Red Hat OpenShift by using the instructions in the Red Hat OpenShift documentation . Information about installing a cluster in a restricted network is given in Mirroring images for a disconnected installation .

2. Install the Red Hat OpenShift command line interface (oc) on your cluster's boot node and run oc login. For more information, see the instructions in Getting started with the Red Hat OpenShift CLI.

3. Optionally configure a custom certificate for Infrastructure Automation to use. You can use either of the following methods:

   • Configure a custom certificate for the Red Hat OpenShift cluster. Follow the instructions in the Red Hat OpenShift documentation Replacing the default ingress certificate. Then, deploy the signing CA certificate into the cluster by following the instructions in the Red Hat OpenShift documentation Replacing the CA Bundle certificate.
   • If you would like to use a custom certificate for Infrastructure Automation only, then after installation is complete follow the instructions in Using a custom certificate.

1.2 Set up a target registry

You must have a local Docker type production-grade registry available in the air-gapped environment to store the Infrastructure Automation images in. The registry must meet the following requirements:

• supports Docker Manifest V2, Schema 2.
• supports multi-architecture images.
• is accessible from the Red Hat OpenShift cluster nodes.
• allows path separators in the image name.
• you have the username and password for a user who can read from and write to the registry.
• must have 59 GB of storage to hold all of the software that is to be transferred to the target registry.

If you do not already have a suitable production-grade registry available, then you must install and configure one. For more information, see About the mirror registry in the Red Hat OpenShift documentation.

Important: Do not use the Red Hat OpenShift image registry as your target registry. The Red Hat OpenShift registry does not support multi-architecture images or path separators in the image name.

1.3 Prepare a host

Regardless of which type of host you're using, you must be able to connect it to the internet and to the air-gapped network with access to the Red Hat® OpenShift® Container Platform cluster and the local, intranet Docker type registry. Your host must be on a Linux® x86_64 or Mac platform with any operating system that the IBM Cloud Pak® CLI and the Red Hat® OpenShift® Container Platform CLI support. If you are on a Windows platform, you must run the actions in a Linux® x86_64 VM or from a Windows Subsystem for Linux (WSL) terminal.

Your host must have 59 GB of storage to hold all of the software that is to be transferred to the local, intranet Docker type registry.

The following table explains the software requirements for installing Infrastructure Automation in an air-gapped environment:

Complete the following steps on your host:

1. Install OpenSSL version 1.11.1 or higher.

2. Install Docker or Podman.

   • To install Docker (for example, on Red Hat® Enterprise Linux®), run the following commands:

     yum check-update
     yum install docker
     

     Note: Docker is not shipped or supported by Red Hat for Red Hat Enterprise Linux (RHEL) 8. The podman container engine replaced docker as the preferred, maintained, and supported container runtime of choice for Red Hat Enterprise Linux 8 systems. For more information, see the Red Hat documentation Running containers without Docker .

   • To install Podman, see Podman Installation Instructions .

3. Install httpd-tools.

   yum install httpd-tools
   

4. Install the oc Red Hat® OpenShift® Container Platform CLI tool. For more information, see Install the OpenShift CLI.

5. Create a directory that serves as the offline store.

   Following is an example directory, which is used in the subsequent steps.

   mkdir $HOME/offline
   

   Note: This offline store must be persistent to avoid transferring data more than once. The persistence also helps to run the mirroring process multiple times or on a schedule.

The portable device must be configured such that it can independently connect to the internet and to the air-gapped network.

1.4 Install the IBM Catalog Management Plug-in for IBM Cloud Pak®

The IBM Catalog Management Plug-in for IBM Cloud Pak (ibm-pak-plugin) is used for the deployment of IBM Cloud Paks® in a disconnected environment. It simplifies the process for discovering required IBM product images and uses standard tools for registry and cluster access. The ibm-pak-plugin also extends the Red Hat OpenShift CLI (oc) capability to streamline the process of delivering installation images to the IBM Cloud Pak in an air-gapped environment.

1. Download and install the most recent version of the ibm-pak-plugin for your host operating system from github.com/IBM.

2. Run the following command to extract the files.

   tar -xf oc-ibm_pak-linux-amd64.tar.gz
   

3. Run the following command to move the file to the /usr/local/bin directory.

   mv oc-ibm_pak-linux-amd64 /usr/local/bin/oc-ibm_pak
   

   Note: If you are installing as a non-root user, you must use sudo.

4. Confirm that the ibm-pak-plugin is installed by running the following command.

   oc ibm-pak --help
   

   Expected result: The ibm-pak-plugin usage is displayed.

1.5 Download documentation and scripts for offline access

Download the following documents and scripts that you will need to be able to access offline during your Infrastructure Automation installation or uninstallation, and copy them to your air-gapped environment.

3.1. Generate mirror manifests

Run the following command to generate mirror manifests to be used when mirroring the images to the target registry.

oc ibm-pak generate mirror-manifests $IA_CASE_NAME file://cpfs --version $IA_CASE_VERSION --final-registry $TARGET_REGISTRY/cpfs

A new directory named ~/.ibm-pak/mirror is created when you issue the oc ibm-pak generate mirror-manifests command. The files images-mapping.txt and image-content-source-policy.yaml are generated at ~/.ibm-pak/data/mirror/$IA_CASE_NAME/$IA_CASE_VERSION.

The command will generate the following files at ~/.ibm-pak/data/mirror/$IA_CASE_NAME/$IA_CASE_VERSION:

• images-mapping-to-filesystem.txt
• images-mapping-from-filesystem.txt
• image-content-source-policy.yaml

3.2. Authenticate with the registries

Log in to the registries to generate an authentication file containing the registry credentials, and then create an environment variable that has the location of the authentication file. This file is used later to enable the oc image mirror command to pull the images from the IBM Entitled Registry, and push them to the target registry.

1. Get the authentication credentials for the IBM Entitled Registry.

   1. To obtain the entitlement key that is assigned to your IBMid, log in to MyIBM Container Software Library with the IBMid and password details that are associated with the entitled software.

   2. In the Entitlement keys section, select Copy key to copy the entitlement key.

2. Run the following command to create an environment variable containing your entitlement key.

   export ENTITLED_REGISTRY_PASSWORD=<key>
   

   Where <key> is the entitlement key that you copied in the previous step.

3. Store the authentication credentials for the IBM Entitled Registry and the target registry.

   If you are using Podman, run the following commands:

   podman login cp.icr.io -u cp -p $ENTITLED_REGISTRY_PASSWORD
   podman login $TARGET_REGISTRY -u $TARGET_REGISTRY_USER -p $TARGET_REGISTRY_PASSWORD
   export REGISTRY_AUTH_FILE=${XDG_RUNTIME_DIR}/containers/auth.json
   unset ENTITLED_REGISTRY_PASSWORD
   

   Note: The authentication file is usually at ${XDG_RUNTIME_DIR}/containers/auth.json. For more information, see the Options section in the Podman documentation.

   If you are using Docker, run the following commands:

   docker login cp.icr.io -u cp -p $ENTITLED_REGISTRY_PASSWORD
   docker login $TARGET_REGISTRY -u $TARGET_REGISTRY_USER -p $TARGET_REGISTRY_PASSWORD
   export REGISTRY_AUTH_FILE=$HOME/.docker/config.json
   unset ENTITLED_REGISTRY_PASSWORD
   

   Note: The authentication file is usually at $HOME/.docker/config.json. For more information, see the Docker documentation

3.3. Mirror the images to the file system

Complete these steps to mirror the images from the internet to a file system on your portable device.

1. Create an environment variable to store the location of the file system where the images are to be stored.

   export IMAGE_PATH=<image-path>
   

   Where <image-path> is the directory where you want the images to be stored.

2. Run the following command to mirror the images from the IBM Entitled Registry to the file system.

   nohup oc image mirror \
   -f ~/.ibm-pak/data/mirror/$IA_CASE_NAME/$IA_CASE_VERSION/images-mapping-to-filesystem.txt \
   -a $REGISTRY_AUTH_FILE \
   --filter-by-os '.*' \
   --insecure \
   --skip-multiple-scopes \
   --dir "$IMAGE_PATH" \
   --max-per-registry=1 > my-mirror-progress.txt 2>&1 &
   

   The UNIX command nohup is used to ensure that the mirroring process continues even if there is a loss of network connection, and redirection of output to a file provides improved monitoring and error visibilty.

   Run the following command if you want to see the progress of the mirroring:

   tail -f my-mirror-progress.txt
   

   Note: If an error occurs during mirroring, the mirror command can be re-run.

3.4 Setup the file system in the air-gapped environment

1. Disconnect the device which has your file system, (the portable compute device or the local compute device) from the internet and connect it to the air-gapped environment.

2. Ensure that environment variables are set on the device in the air-gapped environment that has access to the target registry.

   If you are using a portable storage device, then set the following environment variables on your local compute device within the air-gapped environment.

   If you are using a portable compute device which you have restarted since mirroring the images, then your environment variables will have been lost and you will need to set the following environment variables on your portable compute device again.

   export CASE_NAME=ibm-cp-waiops
   export CASE_VERSION=1.5.2
   export TARGET_REGISTRY_HOST=<IP_or_FQDN_of_target_registry>
   export TARGET_REGISTRY_PORT=<port_number_of_target_registry>
   export TARGET_REGISTRY=$TARGET_REGISTRY_HOST:$TARGET_REGISTRY_PORT
   export TARGET_REGISTRY_USER=<username>
   export TARGET_REGISTRY_PASSWORD=<password>
   export IMAGE_PATH=<image_path>
   

3.5 Authenticate with the target registry

Authenticate with the target registry in the air-gapped environment that you will be mirroring the images into.

If you are using Podman, run the following command:

podman login $TARGET_REGISTRY -u $TARGET_REGISTRY_USER -p $
export REGISTRY_AUTH_FILE=${XDG_RUNTIME_DIR}/containers/auth.json

Note: The authentication file is usually at ${XDG_RUNTIME_DIR}/containers/auth.json. For more information, see the Options section in the Podman documentation.

If you are using Docker, run the following command:

docker login $TARGET_REGISTRY -u $TARGET_REGISTRY_USER -p $TARGET_REGISTRY_PASSWORD
export REGISTRY_AUTH_FILE=$HOME/.docker/config.json

Note: The authentication file is usually at $HOME/.docker/config.json. For more information, see the Docker documentation

3.6 Mirror the images to the target registry from the file system

Complete the steps in this section on the device which has your file system (the portable compute device or the local compute device) to copy the images from the file system to the $TARGET_REGISTRY. Your device with the file system must be connected to both the target registry and the Red Hat OpenShift cluster.

Run the following command to copy the images referenced in the images-mapping-from-filesystem.txt from the $IMAGE_PATH file system to the final target registry.

nohup oc image mirror \
-f ~/.ibm-pak/data/mirror/$IA_CASE_NAME/$IA_CASE_VERSION/images-mapping-from-filesystem.txt \
--from-dir "$IMAGE_PATH" \
-a $REGISTRY_AUTH_FILE \
--filter-by-os '.*' \
--insecure  \
--skip-multiple-scopes \
--max-per-registry=1 > my-mirror-progress2.txt 2>&1 &

The UNIX command nohup is used to ensure that the mirroring process continues even if there is a loss of network connection, and redirection of output to a file provides improved monitoring and error visibilty.

Run the following command if you want to see the progress of the mirroring:

tail -f my-mirror-progress2.txt

Note: If an error occurs during mirroring, the mirror command can be re-run.

3.7 Configure the cluster

1. Log in to your OpenShift cluster.

   You can identify your specific oc login command by clicking the user menu in the upper left corner of the Red Hat OpenShift console, and then clicking Copy Login Command.

   Example:

   oc login <server> -u <cluster username> -p <cluster pass>
   

2. Update the global image pull secret for your Red Hat OpenShift cluster.

   Follow the steps in the Red Hat OpenShift documentation topic Updating the global cluster pull secret .

   These steps enable your cluster to have authentication credentials in place to pull images from your TARGET_REGISTRY as specified in the image-content-source-policy.yaml, which you will apply to your cluster in the next step.

3. Create the ImageContentSourcePolicy.

   Run the following command to create an ImageContentSourcePolicy (ICSP).

   oc apply -f  ~/.ibm-pak/data/mirror/$IA_CASE_NAME/$IA_CASE_VERSION/image-content-source-policy.yaml
   

4. Verify that the ImageContentSourcePolicy resource is created.

   oc get imagecontentsourcepolicy
   

5. Verify your cluster node status.

   oc get MachineConfigPool -w
   

   Important: After the ImageContentsourcePolicy and global image pull secret are applied, the configuration of your nodes will be updated sequentially. Wait until all of the MachineConfigPools are updated before proceeding to the next step.

6. (Optional) If you use an insecure registry, you must add the target registry to the cluster's insecureRegistries list.

   oc patch image.config.openshift.io/cluster --type=merge \
   -p '{"spec":{"registrySources":{"insecureRegistries":["'${TARGET_REGISTRY}'"]}}}'
   

5.1 Create a custom project (namespace)

Create a project (namespace) called cp4waiops for your Infrastructure Automation deployment, by running the following command:

oc create namespace cp4waiops

5.2 Create the catalog source

1. Set the project (namespace) to install the catalog:

   export CATALOG_NAMESPACE=openshift-marketplace
   

2. Create the catalog source.

   oc ibm-pak launch \
     $IA_CASE_NAME \
     --version $IA_CASE_VERSION \
     --action install-catalog \
     --inventory $IA_CASE_INVENTORY_SETUP \
     --namespace $CATALOG_NAMESPACE \
     --args "--registry $TARGET_REGISTRY --recursive \
     --inputDir ~/.ibm-pak/data/cases/$IA_CASE_NAME/$IA_CASE_VERSION"
   

5.3 Create an air-gapped instance of Infrastructure Automation

Follow the instructions in Online production install of IBM Cloud Pak for Watson AIOps Infrastructure Automation from step 6, Install the Infrastructure Automation operator onwards.