Preparing to install IBM Netcool Operations Insight on Red Hat OpenShift in an air-gapped environment (offline) with a portable compute device

Edit online

If your cluster is not connected to the internet, you can deploy a production install of IBM® Netcool® Operations Insight® on Red Hat® OpenShift® on your cluster by using a portable compute device.

You can also deploy IBM Netcool Operations Insight on Red Hat OpenShift in an air-gapped environment by using a bastion host or a portable compute device. For more information, see Installing Netcool Operations Insight in an air-gapped environment (offline) with a bastion host and Preparing to install IBM Netcool Operations Insight on Red Hat OpenShift in an air-gapped environment (offline) with a portable storage device.

Before you begin

Review the Preparing section. Your environment must meet the system requirements.

The operator images for Netcool Operations Insight on Red Hat OpenShift are in the freely accessible operator repository (icr.io/cpopen), and the operand images are in the IBM Entitled Registry (cp.icr.io), for which you require an entitlement key. The CASE bundle is available from IBM cloudPaks. Ensure that you have access to icr.io:443 for the IBM Entitled Registry and Netcool Operations Insight on Red Hat OpenShift catalog source.

For more information about the OLM, see https://access.redhat.com/documentation/en-us/openshift_container_platform/4.10/html/operators/understanding-operators#operator-lifecycle-manager-olm external link .

Ensure that you have access to quay.io:443 for the Netcool Operations Insight on Red Hat OpenShift catalog and images. The quay.io/openshift-release-dev/ocp-v4.0-art-dev image is required and must be present in the cluster to install the NOI operator.

Important: The following procedure is based on a Red Hat OpenShift 4.8 environment and includes links for that version. If your environment uses a different supported version of OpenShift, ensure that you follow the OpenShift documentation for that version.

Prepare a host

Complete the following steps on your host:

Install OpenSSL 1.11.1 or later. For more information, see Downloads in the OpenSSL documentation.
Install Docker or Podman.
- To install Docker (for example, on Red Hat Enterprise Linux®), run the following commands:
```
yum check-update
yum install docker
```
  Note: Docker is not shipped or supported by Red Hat for Red Hat Enterprise Linux (RHEL) 8. The podman container engine replaced docker as the preferred, maintained, and supported container runtime of choice for Red Hat Enterprise Linux 8 systems. For more information, see Running containers without Docker in the Red Hat documentation.
- To install Podman, see Podman Installation Instructions in the Podman documentation.
Install httpd-tools.
```
yum install httpd-tools
```
Download and install the most recent version of cloudctl-linux-amd64.tar.gz from the IBM/cloud-pak-cli repo.
Note: If you want to install a cloudctl version greater than version 3.9, set the following environment variable, or your CASE actions will fail later on.
```
export CLOUDCTL_LAUNCH_SKIP_PREREQ_CHECK=true
```
1. Extract the binary file by entering the following command:
```
tar -xf cloudctl-linux-amd64.tar.gz
```
2. Run the following command to modify the file:
```
chmod 755 cloudctl-linux-amd64
```
3. Run the following command to move the file to the /usr/local/bin directory:
```
mv cloudctl-linux-amd64 /usr/local/bin/cloudctl
```
  Note: You can confirm that cloudctl is installed by entering the cloudctl --help command.
The cloudctl usage is displayed.
Install the oc Red Hat OpenShift CLI tool. For more information, see Install the OpenShift CLI.
Install the skopeo CLI 1.0.0 or higher. For more information, see install.md in the skopeo repo .
Create a directory that serves as the offline store. The following directory is an example directory, which is used in subsequent steps.
```
mkdir $HOME/offline
```
Note: This offline store must be persistent to avoid transferring data more than once. The persistence also helps to run the mirroring process multiple times or on a schedule.

The portable device must be configured such that it can independently connect to the internet and to the air-gapped network.

Download the case bundle

Set the following variables:

export CASE_NAME=ibm-netcool-prod
export CASE_VERSION=1.4.0
export CASE_ARCHIVE=$CASE_NAME-$CASE_VERSION.tgz
export CASE_INVENTORY_SETUP=noiOperatorSetup
export OFFLINEDIR=$HOME/offline
export OFFLINEDIR_ARCHIVE=offline.tgz
export CASE_REPO_PATH=https://github.com/IBM/cloud-pak/raw/master/repo/case
export CASE_LOCAL_PATH=$OFFLINEDIR/$CASE_ARCHIVE
export PORTABLE_DOCKER_REGISTRY_HOST=localhost
export PORTABLE_DOCKER_REGISTRY_PORT=443
export PORTABLE_DOCKER_REGISTRY=$PORTABLE_DOCKER_REGISTRY_HOST:$PORTABLE_DOCKER_REGISTRY_PORT
export PORTABLE_DOCKER_REGISTRY_USER=localuser
export PORTABLE_DOCKER_REGISTRY_PASSWORD=l0calPassword!
export PORTABLE_DOCKER_REGISTRY_PATH=$OFFLINEDIR/imageregistry
export IBM_ENTITLEMENT_KEY=

Download the Netcool Operations Insight CASE bundle (ibm-netcool-prod), into a local directory on your portable compute device.

cloudctl case save \
  --case $CASE_NAME \
  --version $CASE_VERSION \
  --outputdir $OFFLINEDIR \
  --repo https://github.com/IBM/cloud-pak/raw/master/repo/case

Verify that the Netcool Operations Insight CASE bundle, images.csv, and charts.csv are successfully downloaded by running the following command:
```
find $OFFLINEDIR -type f
```

Setting up the portable docker registry

Run a Docker registry service from your connected device (localhost) by completing the following steps:

Open your CLI and go to your $HOME directory. Initialize the Docker type registry by running the following command:

cloudctl case launch \
  --case $CASE_LOCAL_PATH \
  --inventory $CASE_INVENTORY_SETUP \
  --action init-registry \
  --args "--registry $PORTABLE_DOCKER_REGISTRY_HOST --port $PORTABLE_DOCKER_REGISTRY_PORT --user $PORTABLE_DOCKER_REGISTRY_USER --pass $PORTABLE_DOCKER_REGISTRY_PASSWORD --dir $PORTABLE_DOCKER_REGISTRY_PATH"

Start the Docker type registry by running the following command:

cloudctl case launch \
  --case $CASE_LOCAL_PATH \
  --inventory $CASE_INVENTORY_SETUP \
  --action start-registry \
  --args "--registry $PORTABLE_DOCKER_REGISTRY_HOST --port $PORTABLE_DOCKER_REGISTRY_PORT --user $PORTABLE_DOCKER_REGISTRY_USER --pass $PORTABLE_DOCKER_REGISTRY_PASSWORD --dir $PORTABLE_DOCKER_REGISTRY_PATH"

Your host is now configured and you are ready to mirror your images.

Mirroring images to the portable docker registry

Set up access to the IBM Entitled Registry, cp.icr.io, which you pull images from. Run the following commands on your portable compute device.

cloudctl case launch \
    --case $CASE_LOCAL_PATH \
    --inventory noiOperatorSetup \
    --action configure-creds-airgap \
    --args "--registry cp.icr.io --user cp --pass $IBM_ENTITLEMENT_KEY"

cloudctl case launch \
    --case $CASE_LOCAL_PATH \
    --inventory noiOperatorSetup \
    --action configure-creds-airgap \
    --args "--registry $PORTABLE_DOCKER_REGISTRY --user $PORTABLE_DOCKER_REGISTRY_USER --pass $PORTABLE_DOCKER_REGISTRY_PASSWORD"

Before mirroring images, set CLOUDCTL_CASE_USE_CATALOG_DIGEST by running the following command:
```
export CLOUDCTL_CASE_USE_CATALOG_DIGEST=1
```

Mirror the images from IBM Entitled registry to the portable docker registry by running the following commands

cloudctl case launch \
    --case $CASE_LOCAL_PATH \
    --inventory noiOperatorSetup \
    --action mirror-images \
    --args "--registry $PORTABLE_DOCKER_REGISTRY --inputDir $OFFLINEDIR --chunks 14"

Copying the offline data to the portable compute device

Save the registry image.

For a Podman registry, run the following command to save the image:

podman save -o $PORTABLE_DOCKER_REGISTRY_PATH/registry-image.tar docker.io/library/registry:2.6

For a Docker registry, run the following command to save the image:

docker save -o $PORTABLE_DOCKER_REGISTRY_PATH/registry-image.tar docker.io/library/registry:2.6

Copy the saved offline data for a portable compute device. Connect the portable compute device, such as a laptop, to the external host (connected compute device). Archive the offline data for transfer by running the following command:
```
tar -cvzf $OFFLINEDIR_ARCHIVE -C $OFFLINEDIR .
```
Copy the compressed file to the portable compute device.

Setting up the local docker registry

Run the following steps on the server with access to Red Hat OpenShift and your local docker registry.

Set up the following environment variables:

export CASE_NAME=ibm-netcool-prod
export CASE_VERSION=1.4.0
export CASE_ARCHIVE=$CASE_NAME-$CASE_VERSION.tgz
export CASE_INVENTORY_SETUP=noiOperatorSetup
export OFFLINEDIR=$HOME/offline
export OFFLINEDIR_ARCHIVE=offline.tgz
export CASE_REPO_PATH=https://github.com/IBM/cloud-pak/raw/master/repo/case
export CASE_LOCAL_PATH=$OFFLINEDIR/$CASE_ARCHIVE
export PORTABLE_DOCKER_REGISTRY_HOST=localhost
export PORTABLE_DOCKER_REGISTRY_PORT=443
export PORTABLE_DOCKER_REGISTRY=$PORTABLE_DOCKER_REGISTRY_HOST:$PORTABLE_DOCKER_REGISTRY_PORT
export PORTABLE_DOCKER_REGISTRY_USER=localuser
export PORTABLE_DOCKER_REGISTRY_PASSWORD=l0calPassword!
export PORTABLE_DOCKER_REGISTRY_PATH=$OFFLINEDIR/imageregistry
export LOCAL_DOCKER_REGISTRY_HOST=<IP_or_FQDN_of_local_docker_registry>
export LOCAL_DOCKER_REGISTRY_PORT=<port_number_of_local_docker_registry>
export LOCAL_DOCKER_REGISTRY=$LOCAL_DOCKER_REGISTRY_HOST:$LOCAL_DOCKER_REGISTRY_PORT
export LOCAL_DOCKER_REGISTRY_USER=<username>
export LOCAL_DOCKER_REGISTRY_PASSWORD=<password>
export TARGET_NAMESPACE=<namespace-for-noi>

Extract the transferred offline data from the portable compute device:

mkdir -p $OFFLINEDIR 
tar -xvf $OFFLINEDIR_ARCHIVE -C $OFFLINEDIR

Set up the local docker registry. Run the local docker registry as a container. The registry then points to the docker file system directory that was transferred from the external host:

cloudctl case launch \
  --case $CASE_LOCAL_PATH \
  --inventory $CASE_INVENTORY_SETUP \
  --action start-registry \
  --args "--registry $PORTABLE_DOCKER_REGISTRY_HOST --port $PORTABLE_DOCKER_REGISTRY_PORT --user $PORTABLE_DOCKER_REGISTRY_USER --pass $PORTABLE_DOCKER_REGISTRY_PASSWORD --dir $PORTABLE_DOCKER_REGISTRY_PATH"

Configure an authentication secret for the local docker registry.

cloudctl case launch \
  --case $CASE_LOCAL_PATH \
  --inventory $CASE_INVENTORY_SETUP \
  --action configure-creds-airgap \
  --args "--registry $PORTABLE_DOCKER_REGISTRY --user $PORTABLE_DOCKER_REGISTRY_USER --pass $PORTABLE_DOCKER_REGISTRY_PASSWORD"

Store credentials of the registry that is going to serve images to the cluster or workloads:

cloudctl case launch \
  --case $CASE_LOCAL_PATH \
  --inventory $CASE_INVENTORY_SETUP \
  --action configure-creds-airgap \
  --args "--registry $LOCAL_DOCKER_REGISTRY --user $LOCAL_DOCKER_REGISTRY_USER --pass $LOCAL_DOCKER_REGISTRY_PASSWORD"

Before mirroring images, set CLOUDCTL_CASE_USE_CATALOG_DIGEST by running the following command:
```
export CLOUDCTL_CASE_USE_CATALOG_DIGEST=1
```

Mirror the images from the portable registry to the target registry on your cluster:

cloudctl case launch \
  --case $CASE_LOCAL_PATH \
  --inventory $CASE_INVENTORY_SETUP \
  --action mirror-images \
  --args "--fromRegistry $PORTABLE_DOCKER_REGISTRY --registry $LOCAL_DOCKER_REGISTRY --inputDir $OFFLINEDIR --chunks 14"

Configure Red Hat OpenShift cluster for airgap.

Warning:

Cluster resources must adjust to the new pull secret, which can temporarily limit the usability of the cluster. Authorization credentials are stored in $HOME/.airgap/secrets and /tmp/airgap* to support this action.
Applying ImageContentSourcePolicy causes cluster nodes to recycle for OpenShift versions 4.7 or earlier.
Applying ImageContentSourcePolicy causes cluster nodes to temporarily have a STATUS SchedulingDisabled status for OpenShift versions 4.8 or later.

Configure your Red Hat OpenShift cluster for airgap. This step can take 90+ minutes, if nodes are recycled.

Insecure registry

If your offline registry is insecure, complete the following steps for registry access:

oc login -u kubeadmin -p <password> <server's REST API URL>

Run the following command on your bastion server to create a global image pull secret for the target registry, and create an ImageContentSourcePolicy.
```
oc patch image.config.openshift.io/cluster --type=merge -p '{"spec":{"registrySources":{"insecureRegistries":["'${LOCAL_DOCKER_REGISTRY}'"]}}}'
```

Verify that the ImageContentSourcePolicy is created successfully, by running the oc get MachineConfigPool command. Confirm that the value in the UPDATED column is set to true, as in the following example:

oc get MachineConfigPool
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-cd77bc65ebc2442ad4336bca7b61a6be   True      False      False      3              3                   3                     0                      25h
worker   rendered-worker-6418363c7fa863367edecf08eca421e4   True      False      False      4              4                   4                     0                      25h

Secure registry

If your offline registry is secure, complete the following steps for registry access. You can add certificate authorities (CA) to the cluster for use when pushing and pulling images. You must have cluster administrator privileges. You must have access to the public certificates of the registry, usually a hostname/ca.crt file located in the /etc/docker/certs.d/ directory.

Create a configmap in the openshift-config namespace that contains the trusted certificates for the registries that use self-signed certificates. For each CA file, ensure that the key in the configmap is the hostname of the registry, in the hostname[..port] format:
```
oc create configmap registry-cas -n openshift-config \
--from-file=myregistry.corp.com..5000=/etc/docker/certs.d/myregistry.corp.com:5000/ca.crt \
```

Update the cluster image configuration:

oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"registry-cas"}}}' --type=merge

Run the following commands for your secure or insecure registry:

oc create namespace $TARGET_NAMESPACE

cloudctl case launch \
  --case $OFFLINEDIR/$CASE_ARCHIVE \
  --namespace $TARGET_NAMESPACE \
  --inventory noiOperatorSetup \
  --action configure-cluster-airgap \
  --args "--registry $LOCAL_DOCKER_REGISTRY --inputDir $OFFLINEDIR --user $LOCAL_DOCKER_REGISTRY_USER --pass $LOCAL_DOCKER_REGISTRY_PASSWORD"

Monitor the machineconfigpool pod to verify that the update is completely pushed to all nodes. Ensure that the UPDATED column displays True for master and worker nodes, as in the following example:

oc get machineconfigpool
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-f4b89d6ca38d7b11a8a2decaf59ba6dd   True      False      False      3              3                   3                     0                      13h
worker   rendered-worker-75afdaafd9615c58a165b118fef04750   True      False      False      4              4                   4                     0                      13h

Install the Netcool Operations Insight catalog and operator.

Install the catalog by using CASE:

Run the following command:

oc create secret docker-registry target-registry-secret \
    --docker-server=$LOCAL_DOCKER_REGISTRY \
    --docker-username=$LOCAL_DOCKER_REGISTRY_USER \
    --docker-password=$LOCAL_DOCKER_REGISTRY_PASSWORD \
    --namespace=$TARGET_NAMESPACE

Set the project (namespace) to install the catalog:

export CATALOG_NAMESPACE=openshift-marketplace

Create the catalog source:

cloudctl case launch \
  --case $CASE_LOCAL_PATH \
  --inventory $CASE_INVENTORY_SETUP \
  --action install-catalog \
  --namespace $CATALOG_NAMESPACE \
  --args "--registry $LOCAL_DOCKER_REGISTRY/cpopen"

Verify the Netcool Operations Insight catalog source. From the Red Hat OpenShift OLM UI, go to Administration > Cluster Settings, and then select the OperatorHub configuration resource under the Global Configuration tab. Verify that the ibm-noi-catalog catalog source is present.
Install the Netcool Operations Insight operator by using CASE. Run the following command:
```
cloudctl case launch \
    --case $CASE_LOCAL_PATH \
    --namespace $TARGET_NAMESPACE \
    --inventory $CASE_INVENTORY_SETUP \
    --action install-operator \
    --args "--secret target-registry-secret"
```
Where:
- $TARGET_NAMESPACE is the custom namespace to be used for your deployment.
- target-registry-secret is the secret for accessing the target registry that you created previously.
Verify the status of IBM Cloud Pak for Watson™ AIOps Event Manager. From the Red Hat OpenShift OLM UI, go to Operators > Installed Operators, and verify that the status of IBM Cloud Pak for Watson AIOps Event Manager is Succeeded.

Create a Netcool Operations Insight instance.

From the Red Hat OpenShift OLM UI, go to Operators > Installed Operators, and select IBM Cloud Pak for Watson AIOps Event Manager. Under Provided APIs > NOI, select Create Instance.
From the Red Hat OpenShift OLM UI, use the YAML view or the Form view to configure the properties for the Netcool Operations Insight deployment. For more information about configurable properties for a cloud-only deployment, see Cloud operator properties.
Note: Ensure that the name of the Netcool Operations Insight instance does not exceed 10 characters.
Edit the Netcool Operations Insight properties to provide access to the target registry. Set spec.entitlementSecret to the target registry secret.
Select Create.
Under the All Instances tab, a Netcool Operations Insight instance appears. To monitor the status of the installation, see Monitoring cloud installation progress.
Note:
- Changing an existing deployment from a Trial deployment type to a Production deployment type is not supported.
- Changing an instance's deployment parameters in the Form view is not supported post deployment.
- If you update custom secrets in the OLM console, the crypto key is corrupted and the command to encrypt passwords does not work. Update custom secrets only with the CLI. For more information about storing a certificate as a secret, see https://www.ibm.com/docs/en/SS9LQB_1.1.13/LoadingData/t_asm_obs_configuringsecurity.html

What to do next

To enable or disable a feature or observer after installation edit the Netcool Operations Insight instance by running the command:
```
oc edit noi <noi-instance-name>
```
Where <noi-instance-name> is the name of the deployment that you want to change.
You can then select to enable or disable the feature or observer. When you disable features postinstallation , the resource is not automatically deleted. To find out whether the feature is deleted, you must check the asm-operator log.