Mirroring the IBM App Connect images to an air-gapped cluster

Kubernetes-only content In production systems, it is common to have a cluster that does not have internet access. In these cases, you can install the IBM® App Connect Operator and Kubernetes in an air-gapped environment, which is also referred to as an offline, restricted, or disconnected environment. Air-gapped installations require you to mimic a typical online installation by using images in your own registry.

To make the IBM App Connect Operator available for installation in your air-gapped cluster, you need to first mirror the images for the Operator to your own private or local registry. When you install the Operator, its images are pulled from your private registry.

Before you attempt to mirror the images, you must set up a host that can access both the public internet and the air-gapped environment where a private (local) registry and Kubernetes clusters reside. You can then use this host to mirror your images directly to your private registry by using a supplied script.

Tip: If you want to install the IBM App Connect Operator from an online cluster, skip this task and move on to Installing the IBM App Connect Operator by using a Helm chart.

Step 1. Setting up your mirroring environment

To set up a host that can be connected to the internet to mirror your environment, complete the following steps:

Prerequisites for your mirroring environment
Preparing a host for mirroring the images
Setting up the private (local) image registry and access

Prerequisites for your mirroring environment

Your cluster must be set up and running Kubernetes 1.27.x, 1.28.x, 1.29.x, 1.30.x, or 1.31.x.
You must have a Docker V2 registry that is available and accessible from the Kubernetes cluster nodes. (For more information, see Setting up the private (local) image registry and access.)
The following sites and ports must be accessible:
- icr.io:443 for IBM Entitled Registry
- quay.io:443 for IBM App Connect
- github.com for the Helm chart and tools
Storage must be available and configured on your cluster.

Preparing a host for mirroring the images

Your host must have access to a set of resources and command-line tools.

You must be able to connect your host to the internet (to download the images) and to the air-gapped environment at the same time. In the air-gapped environment, the host must be able to access the Kubernetes cluster (to install the Operator) and the private registry (where the images will be mirrored).
Your host must be on a Linux® x86_64 or Mac platform with any operating system that the Kubernetes CLI supports.
Important: If you are using Windows, execute the actions in a Linux x86_64 VM or from a Windows Subsystem for Linux (WSL) terminal.

Setting up the private (local) image registry and access

Use a local Docker registry to store your images in your air-gapped environment. If you already have one or more centralized, corporate registry servers that store production container images, you can use one of them as your local registry. If a registry is not already available, install and configure a production-grade registry.

The local registry must meet the following requirements:

Supports multi-architecture images through Docker Manifest V2, Schema 2.
Has sufficient storage to hold all the software that is to be transferred to the local registry.
Is accessible from the Kubernetes cluster nodes.
Allows path separators in the image name.
Supports auto-repository creation.

The user access requirements for this registry are as follows:

To access your registries during the mirroring process, you need user credentials that can write to the target local registry and create repositories.
To access your registries during run time, you need user credentials that can read all repositories from the target local registry. (The Kubernetes cluster uses these credentials.)

Your host is now configured and you are ready to mirror your images.

Step 2. Mirroring the IBM App Connect images with the mirrorImages.sh script

The process of mirroring images pulls the image from the internet and pushes it to your private (local) registry.

Some of the App Connect images are in the IBM Entitled Registry (which requires authentication), and other images are in IBM Container Registry (which does not need any special credentials). You need to log in to the IBM Entitled Registry for source images, and also log in to your local registry that will be the target when images are copied over. After you mirror your images, you can complete the air-gapped installation.

To mirror the images, complete the following steps:

Prerequisites for mirroring images
Logging in to the IBM Entitled Registry registry
Logging in to the private (local) registry
Downloading the mirrorImages.sh script
Running the mirrorImages.sh script

Prerequisites for mirroring images

This script requires skopeo and yq version 4 to be installed. Support is also available in cases where yq is installed with another binary name (for example, yq4) to enable co-existence with the yq version 3 binary.
You must be logged in to the IBM Entitled Registry. For more information, see Logging in to the IBM Entitled Registry registry.
You must be logged in to the private registry to which you are mirroring your images. For more information, see Logging in to the private (local) registry.

Logging in to the IBM Entitled Registry registry

Before you mirror the images, log in to the IBM Entitled Registry, which acts as a source image registry.

To log in to the IBM Entitled Registry, complete the following steps:

Obtain an entitlement key, which is required to pull App Connect images from the IBM Entitled Registry.
1. Log in the IBM Container software library with the IBMid and password that are associated with the entitled software.
2. To obtain a new key, click Get entitlement key or Add new key, and then click Copy.
  If you have an existing key to use, click Copy.
3. Save the entitlement key for use later.
Log in to the cp.icr.io registry:
```
skopeo login cp.icr.io
```
When prompted, specify cp as your username and your entitlement key (which you copied earlier) as the password. (You can optionally run the command with the -u and -p parameters, as shown in the following example.)
```
skopeo login cp.icr.io -u cp -p myEntitlementKey
```
The output should indicate that your login succeeded.

Logging in to the private (local) registry

Before you mirror the required images, also log in to the local registry to which you are mirroring your images.

To log in to the local registry (for example, my.PrivateRegistry.com), run the following command:

skopeo login my.PrivateRegistry.com

When prompted, specify your username and password. Use an account that can write images to the local registry. (You can optionally run the command with the -u and -p parameters.)

The output should indicate that your login succeeded.

Downloading the mirrorImages.sh script

Download the attached mirrorImages.zip file. This .zip file contains a documentation.md file and a mirrorImages.sh script.

Running the mirrorImages.sh script

Before you run the mirrorImages.sh script, connect your host to the internet. To mirror the images, your host must be connected to both the internet and the air-gapped environment that contains the local registry.

The mirrorImages.sh script has two modes:

copy: This option automatically copies the images from the IBM Entitled Registry into your own private registry by using skopeo.
list: This option generates a file with a list of all the images that need to be mirrored. You can use this option if you want to create your own custom mirroring.

The mirrorImages.sh script accepts four parameters:

copy or list
version: The version of App Connect that you want to mirror
remoteRegistry: The registry that you want to copy the images to
Optional: The name of your yq4 binary

To mirror the App Connect images, run the mirrorImages.sh command with the copy parameter.

./mirrorAppConnect.sh copy version remoteRegistry optional-yq4-binary

This example of the mirrorImages.sh command uses the copy parameter to copy the images to a private registry called my.domain.com/namespace by using the yq4 binary:

./mirrorAppConnect.sh copy 12.11.1 my.domain.com/namespace yq4

The following example shows a sample mirrorImages.sh command with the copy parameter, and the sample output.

./mirrorImages.sh copy 12.11.1 docker-na.artifactory.devops.com/appconnect-airgap-docker-local/k8s yq4

yq binary = yq4
Downloading the case for IBM App Connect version: 12.11.1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 1063k  100 1063k    0     0  1105k      0 --:--:-- --:--:-- --:--:-- 1105k
x ibm-appconnect
x ibm-appconnect/LICENSE
x ibm-appconnect/README.md
x ibm-appconnect/case.yaml
x ibm-appconnect/certifications
x ibm-appconnect/digests.yaml
x ibm-appconnect/inventory
x ibm-appconnect/inventory/ibmAppconnect
x ibm-appconnect/inventory/ibmAppconnect/README.md
x ibm-appconnect/inventory/ibmAppconnect/actions.yaml
x ibm-appconnect/inventory/ibmAppconnect/files
x ibm-appconnect/inventory/ibmAppconnect/files/airgap.sh
x ibm-appconnect/inventory/ibmAppconnect/files/launch-impl.sh
x ibm-appconnect/inventory/ibmAppconnect/files/launch.sh
x ibm-appconnect/inventory/ibmAppconnect/files/op-olm
x ibm-appconnect/inventory/ibmAppconnect/files/op-olm/catalog_source.yaml
x ibm-appconnect/inventory/ibmAppconnect/inventory.yaml
x ibm-appconnect/inventory/ibmAppconnect/resources.yaml
x ibm-appconnect/licenses
x ibm-appconnect/licenses/LICENSE
x ibm-appconnect/licenses/licenses-dev
x ibm-appconnect/licenses/licenses-dev/LICENSE
x ibm-appconnect/licenses/licenses-dev/REDIST
x ibm-appconnect/licenses/licenses-dev/copyright-attributes
x ibm-appconnect/licenses/licenses-dev/non_ibm_license
x ibm-appconnect/licenses/licenses-dev/notices
x ibm-appconnect/licenses/licenses-prod
x ibm-appconnect/licenses/licenses-prod/IBM_data_studio_4.1.3_notices.txt
x ibm-appconnect/licenses/licenses-prod/LICENSE
x ibm-appconnect/licenses/licenses-prod/MQ_9.3_notices_190522.txt
x ibm-appconnect/licenses/licenses-prod/REDIST
x ibm-appconnect/licenses/licenses-prod/copyright-attributes
x ibm-appconnect/licenses/licenses-prod/non_ibm_license
x ibm-appconnect/licenses/licenses-prod/notices
x ibm-appconnect/prereqs.yaml
x ibm-appconnect/roles.yaml
x ibm-appconnect/signature.yaml
Parsing /tmp/ibm-appconnect/inventory/ibmAppconnect/resources.yaml
Generating list of image in AppConnectImages.txt
Copying icr.io/cpopen/appconnect-operator@sha256:1abcde62fac9c06895a7cb123456e31b06f50ae82b8f7d987cebde2afb5a7766 to docker-na.artifactory.devops.com/appconnect-airgap-docker-local/k8s/appconnect-operator@sha256:1abcde62fac9c06895a7cb123456e31b06f50ae82b8f7d987cebde2afb5a7766
Finished copying image
......
Copying cp.icr.io/cp/appc/acecc-connector-auth-service-prod@sha256:a12345148ff46aa480130b863fcb3ecfaff2861ce66883b45db4ecac68d04abc to docker-na.artifactory.devops.com/appconnect-airgap-docker-local/k8s/acecc-connector-auth-service-prod@sha256:a12345148ff46aa480130b863fcb3ecfaff2861ce66883b45db4ecac68d04abc
Finished copying image

You may now update your helm chart values.yaml with:

  operator:
    deployment:
      repository: docker-na.artifactory.devops.com/appconnect-airgap-docker-local/k8s
  privateRegistry: docker-na.artifactory.devops.com/appconnect-airgap-docker-local/k8s

If not already configured you will also need to specify a pullSecret for your registry in the values.yaml

To generate a file that contains a list of all the App Connect images to be mirrored, run the mirrorImages.sh command with the list parameter. This command generates a file named AppConnectImages-version.txt file, where version is the version value that you specify.

./mirrorAppConnect.sh list version remoteRegistry

This example of the mirrorImages.sh command uses the list parameter to create a file called AppConnectImages-12.11.1.txt with a list of all the images that need to be mirrored:

./mirrorAppConnect.sh list 12.11.1 my.domain.com/namespace

What to do next

Install the Operator in your cluster as described in Installing the IBM App Connect Operator by using a Helm chart.