Mirroring images with a portable compute or storage device (file system)

If your cluster is not connected to the internet, you can mirror images to a registry in your network-restricted environment by using a portable compute or storage device, such as a laptop or USB storage device. This task must be performed by a Red Hat OpenShift administrator.

Download images from the public registries to a local file system. You can then bring the device into the network restricted environment, or transfer the files to a device within the network restricted environment and mirror the images to the local registry.

The overall process for adding catalog sources consists of the following steps:

  1. Set up your mirroring environment

  2. Set environment variables and download CASE files

  3. Mirror the images

Troubleshooting: For a list of possible errors and solutions when performing the following steps, see Troubleshooting issues when mirroring images for an air-gapped cluster.

1. Set up your mirroring environment

Before you install any IBM Cloud Pak® on an air-gapped environment, you must set up a host that can be connected to the internet to complete configuration of your mirroring environment. To set up your mirroring environment, complete the following steps:

1.1 Prerequisites

  • An OpenShift Container Platform 4.12 cluster must be installed.

  • A Docker V2 registry that is available and accessible from the OpenShift Container Platform cluster nodes.

  • The following sites and ports must be accessible:

    • icr.io:443 for entitled registry

    • quay.io:443 for Cloud Pak for Integration

    • github.com for CASEs and tools

    • redhat.com for Red Hat OpenShift Container Platform upgrades

  • Storage is available and configured on your cluster.

1.2 Prepare a host for mirroring the images

You must be able to connect your file system to the internet and to the restricted network environment (with access to the OpenShift Container Platform cluster and the local registry). Your host must be on a Linux® x86_64 or Mac platform with any operating system that the OpenShift Container Platform CLI supports.

Important: If you are on a Windows platform, you must execute the actions in a Linux x86_64 VM or from a Windows Subsystem for Linux (WSL) terminal.

Your file system must have sufficient storage to hold all the software that is to be transferred to the local registry.

The following table provides the software requirements for installing Cloud Pak for Integration in an air-gapped environment:

Software requirements and purpose

Software Purpose
Docker Container management
Podman Container management
oc Red Hat OpenShift Container Platform administration
oc ibm-pak oc IBM Catalog Management Plug-in for IBM Cloud Paks

Complete the following steps on your host:

  1. Install Docker or Podman.

    • To install Docker (for example, on Red Hat® Enterprise Linux®), run the following commands:

    yum check-update
    yum install docker
  2. Install the oc OpenShift Container Platform CLI tool.

    Follow the procedure for Getting started with the OpenShift CLI.

  3. Download the IBM Catalog Management plug-in version 1.1.0 or later from GitHub. This plug-in allows you to run oc ibm-pak commands against the cluster.

    To confirm that ibm-pak is installed, run the following command:

    oc ibm-pak --help

    This should return the oc ibm-pak usage.

1.3. Set up local image registry and access

You must use a local Docker registry to store your images in your network-restricted environment. You may have one or more centralized, corporate registry servers to store production container images. If a local registry is not already available, a production-grade registry will need to be installed and configured.

To access your registries during an air-gapped installation, use an account that can write to the target local registry. To access your registries during runtime, use an account that can read from the target local registry. The registry must meet the following requirements:

  • Supports multi-architecture images through Docker Manifest V2, Schema 2. For details, see Docker Manifest V2, Schema 2.

  • Is accessible from the OpenShift Container Platform cluster nodes.

  • Allows path separators in the image name.

Tip: Do not use OpenShift image registry as your local registry. The OpenShift registry does not support multi-architecture images or path separators in the image name.

After you create the local registry, you must configure the registry, verify that the registry meets the following requirements:

  • Supports auto-repository creation.

  • Has credentials of a user who can write and create repositories. The mirroring process uses these credentials.

  • Has credentials of a user who can read all repositories. The OpenShift Container Platform cluster uses these credentials.

  • Has sufficient storage to hold all the software that is to be transferred to the local registry.

2. Set environment variables and download CASE files

Before mirroring your images, you need to set the environment variables on your mirroring device, and connect to the internet, so that you can download the corresponding CASE files. To finish preparing your host, complete the following steps.

Prerequisites (if you are applying fix packs prior to an upgrade):

  • Confirm that your operators are running properly.

  • If there are any pending operator updates that require manual approval, approve those before starting this procedure. For more information about why manual approval should not be configured for operator installs, see "Restricting automatic updates with an approval strategy" in Installing the operators by using the Red Hat OpenShift console.

  1. Create the following environment variables with the installer image name and the image inventory on your host.

    The table at the end of this section provides the export commands for all operators.

    export CASE_NAME=ibm-integration-platform-navigator
    export CASE_VERSION=7.1.3

    You need to separately mirror each component of Cloud Pak for Integration that you intend to use. CASE files for capabilities can be found in the IBM CASE repository.

  2. Connect your host to the internet. It does not need to be connected to the network restricted environment at this time.

  3. Download the CASE to your host. If you do not specify the CASE version, it downloads the latest version.

    oc ibm-pak get $CASE_NAME --version $CASE_VERSION

Repeat the process for each CASE you are mirroring.

Your host is now configured and you are ready to mirror your images.

Export commands for all operators:

Copy and run the codeblock content in the Export commands column to set the environment variables for a given operator:

Operator Export commands
IBM Cloud Pak for Integration export CASE_NAME=ibm-integration-platform-navigator && export CASE_VERSION=7.1.3
IBM Automation foundation assets export CASE_NAME=ibm-integration-asset-repository && export CASE_VERSION=1.5.20
IBM API Connect (1) export CASE_NAME=ibm-apiconnect && export CASE_VERSION=5.0.0
IBM App Connect export CASE_NAME=ibm-appconnect && export CASE_VERSION=10.1.1
IBM MQ export CASE_NAME=ibm-mq && export CASE_VERSION=2.4.8
IBM Event Streams export CASE_NAME=ibm-eventstreams && export CASE_VERSION=3.2.5
IBM Event Endpoint Management export CASE_NAME=ibm-eventendpointmanagement && export CASE_VERSION=11.0.6
IBM DataPower Gateway (2) export CASE_NAME=ibm-datapower-operator && export CASE_VERSION=1.8.1
IBM Aspera HSTS export CASE_NAME=ibm-aspera-hsts-operator && export CASE_VERSION=1.5.13
IBM Cloud Pak foundational services export CASE_NAME=ibm-cp-common-services && export CASE_VERSION=1.19.13
  • (1) The IBM API Connect CASE also mirrors the IBM DataPower Gateway CASE using the Cloud Pak for Integration image group.

  • (2) The IBM DataPower Gateway CASE contains multiple image groups. To mirror images for Cloud Pak for Integration, use the ibmdpCp4i image group.

3. Mirror the images

The process of mirroring images pulls the image from the internet and saves it to your file system, then pushes the images to your air-gapped environment. After mirroring your images, you can configure your cluster and complete the air-gapped installation.

Complete the following steps to mirror your images from your host to your air-gapped environment:

3.1. Generate mirror manifests

  1. Define the environment variable $TARGET_REGISTRY by running the following command:

    export TARGET_REGISTRY=<target-registry>

    The <target-registry> refers to the registry (hostname and port) where your images will be mirrored to and accessed by the oc cluster. For example: 172.16.0.10:5000.

    If you want the images to use a specific namespace in the target registry, you can specify it here, for example: registry.private/cp4i.

  2. Generate mirror manifests by running the following command:

    oc ibm-pak generate mirror-manifests $CASE_NAME file://integration --version $CASE_VERSION --final-registry $TARGET_REGISTRY

    If you need to filter for a specific image group, add the parameter --filter <image_group> to this command.

This generates the following files at ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION:

  • catalog-sources.yaml

  • catalog-sources-linux-<arch>.yaml (if there are arch specific catalog sources)

  • image-content-source-policy.yaml

  • images-mapping-to-filesystem.txt

  • images-mapping-from-filesystem.txt

Repeat the process for each CASE you are mirroring.

3.2. Authenticate the entitled registry

You must authenticate to the entitled registry to mirror the required images.

  1. Export the path to the file which stores the auth credentials that are generated from podman login or docker login. The authentication file is typically located at $HOME/.docker/config.json on Linux or %USERPROFILE%/.docker/config.json on Windows:

    export REGISTRY_AUTH_FILE=$HOME/.docker/config.json
  2. Login to the cp.icr.io registry with podman or docker:

    podman login cp.icr.io

    See Finding and applying your entitlement key (online installation) for how to obtain your entitlement key.

3.3 Mirror the images to the filesystem

Complete these steps to mirror the images from the internet to your file system:

  1. Define the environment variable $IMAGE_PATH by running the following command:

    export IMAGE_PATH=<image-path>

    The <image-path> refers to the local path to store the images. For example, integration uses the integration directory in the current working directory.

  2. Mirror images to the file system:

    oc image mirror \
      -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping-to-filesystem.txt \
      --filter-by-os '.*' \
      -a $REGISTRY_AUTH_FILE \
      --skip-multiple-scopes \
      --max-per-registry=1 \
      --dir "$IMAGE_PATH"

The oc image mirror command starts by planning what images and layers need to be transferred. It can take a couple of minutes before you start seeing output.

Repeat the process for each CASE you are mirroring. Use the same image path for each CASE so that common images are only stored once.

3.4 Move the file system to the restricted network environment

If you are using a portable device, you can disconnect the device from the internet and connect it to the restricted network environment. The same environment variables can be used.

If you are using portable storage, transfer the files to a device in the restricted network environment. The files that need to be transferred are:

  • The ~/.ibm-pak directory.

  • The image path you specified in the previous step.

If you are using a different device in the restricted network environment, you need to set the same environment variables as the original device. These include CASE_NAME, CASE_VERSION, REGISTRY_AUTH_FILE and IMAGE_PATH.

3.5. Authenticate to the local registry

You must authenticate to the local registry to mirror the required images.

  1. Login to the local registry with podman or docker:

    podman login $TARGET_REGISTRY

    Use an account that can write images to the local registry.

3.6. Mirror the images to the target registry

Run the following command to copy the images to the local registry. Your device must be connected to the restricted network environment that contains the local registry.

oc image mirror \
  -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping-from-filesystem.txt \
  -a $REGISTRY_AUTH_FILE \
  --filter-by-os '.*' \
  --skip-multiple-scopes \
  --max-per-registry=1 \
  --from-dir "$IMAGE_PATH"

The oc image mirror command starts by planning what images and layers need to be transferred. It can take a couple of minutes before you start seeing output.

If the local registry is not secured by TLS, or the certificate presented by the local registry is not trusted by your device, add the --insecure option to the command.

Repeat the process for each CASE you are mirroring.

3.7. Configure the cluster

  1. Log in to your Red Hat OpenShift Container Platform using the oc CLI.

  2. Update the global image pull secret for your Red Hat OpenShift cluster.

    Follow the steps in Updating the global cluster pull secret (use the version selector on the linked page to select v. 4.12). The documented steps in the link enable your cluster to have proper authentication credentials in place to pull images from your TARGET_REGISTRY as specified in the image-content-source-policy.yaml.

  3. Create ImageContentSourcePolicy:

    oc apply -f  ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/image-content-source-policy.yaml

    Verify that the ImageContentSourcePolicy resource is created:

    oc get imageContentSourcePolicy

    Verify your cluster node status and wait for all nodes to be updated before proceeding:

    oc get MachineConfigPool -w