Installing

Creating registry namespaces

Top-level namespaces are the namespaces which appear at the root path of your private registry. For example, if your registry is hosted at myregistry.com:5000, then mynamespace in myregistry.com:5000/mynamespace is defined as a top-level namespace. There can be many top-level namespaces.

When the images are mirrored to your private registry, it is required that the top-level namespace where images are getting mirrored already exists or can be automatically created during the image push. If your registry does not allow automatic creation of top-level namespaces, you must create them manually.

When you generate mirror manifests, you can specify the top-level namespace where you want to mirror the images by setting TARGET_REGISTRY to myregistry.com:5000/mynamespace which has the benefit of needing to create only one namespace mynamespace in your registry if it does not allow automatic creation of namespaces. The top-level namespaces can also be provided in the final registry by using --final-registry.

If you do not specify your own top-level namespace, the mirroring process will use the ones which are specified by the CASEs. For example, it will try to mirror the images at myregistry.com:5000/cp etc.

There can be more top-level namespaces that you might need to create. See section on Generate mirror manifests for information on how to use the oc ibm-pak describe command to list all the top-level namespaces.

Mirroring images to your private container registry

The process of mirroring images takes the image from the internet to your host, then effectively copies that image to your private container registry. After you mirror your images, you can configure your cluster and complete air-gapped installation.

Generate mirror manifests

   oc ibm-pak generate mirror-manifests \
      $CASE_NAME \
      $INTERMEDIATE_REGISTRY \
      --version $CASE_VERSION
      --final-registry $FINAL_REGISTRY

In this case, in place of a single mapping file (images-mapping.txt), two mapping files are created.

1. images-mapping-to-registry.txt
2. images-mapping-from-registry.txt

1. Run the following commands to generate mirror manifests to be used when mirroring from a file system (disconnected mirroring):

   oc ibm-pak generate mirror-manifests \
      $CASE_NAME \
      file://local \
      --final-registry $TARGET_REGISTRY
   

   Example ~/.ibm-pak directory structure for disconnected mirroring
   The following tree shows an example of the ~/.ibm-pak directory structure for disconnected mirroring:

   tree ~/.ibm-pak
   /root/.ibm-pak
   ├── config
   │   └── config.yaml
   ├── data
   │   ├── cases
   │   │   └── ibm-cp-common-services
   │   │       └── 1.9.0
   │   │           ├── XXXX
   │   │           ├── XXXX
   │   └── mirror
   │       └── ibm-cp-common-services
   │           └── 1.9.0
   │               ├── catalog-sources.yaml
   │               ├── image-content-source-policy.yaml
   │               ├── images-mapping-to-filesystem.txt
   │               └── images-mapping-from-filesystem.txt
   └── logs
      └── oc-ibm_pak.log
   

   Note: A new directory ~/.ibm-pak/mirror is created when you issue the oc ibm-pak generate mirror-manifests command. This directory holds the image-content-source-policy.yaml, images-mapping-to-filesystem.txt, images-mapping-from-filesystem.txt, and catalog-sources.yaml files.

   oc ibm-pak generate mirror-manifests \
      ibm-cloud-native-postgresql \
      file://local \
      --final-registry $TARGET_REGISTRY \
      --filter $GROUPS

The command was updated with a --filter argument. For example, for $GROUPS equal to ibmEdbStandard the mirror manifests will be generated only for the images associated with ibm-cloud-native-postgresql in its Standard variant. The resulting image group consists of images in the ibm-cloud-native-postgresql image group as well as any images that are not associated with any groups. This allows products to include common images as well as the ability to reduce the number of images that you need to mirror.

   oc ibm-pak describe $CASE_NAME --version $CASE_VERSION --list-mirror-images

Authenticating the registry

Complete the following steps to authenticate your registries:

1. Store authentication credentials for all source Docker registries.

   Your product might require one or more authenticated registries. The following registries require authentication:

   • cp.icr.io
   • registry.redhat.io
   • registry.access.redhat.com

   You must run the following command to configure credentials for all target registries that require authentication. Run the command separately for each registry:

   Note: The export REGISTRY_AUTH_FILE command only needs to run once.

   export REGISTRY_AUTH_FILE=<path to the file which will store the auth credentials generated on podman login>
   podman login <TARGET_REGISTRY>
   

   Important: When you log in to cp.icr.io, you must specify the user as cp and the password which is your Entitlement key from the IBM Cloud Container Registry. For example:

   podman login cp.icr.io
   Username: cp
   Password:
   Login Succeeded!
   

For example, if you export REGISTRY_AUTH_FILE=~/.ibm-pak/auth.json, then after performing podman login, you can see that the file is populated with registry credentials.

export REGISTRY_AUTH_FILE=$HOME/.docker/config.json

Mirror images to final location

Complete the steps in this section on your host that is connected to both the local Docker registry and the Red Hat® OpenShift® Container Platform cluster.

1. Mirror images to the final location.

   • For mirroring from a bastion host (connected mirroring):

     Mirror images to the TARGET_REGISTRY:

      oc image mirror \
        -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping.txt \
        --filter-by-os '.*'  \
        -a $REGISTRY_AUTH_FILE \
        --insecure  \
        --skip-multiple-scopes \
        --max-per-registry=1 \
        --continue-on-error=true
     

     If you generated manifests in the previous steps to mirror images to an intermediate registry server followed by a final registry server, run the following commands:

     1. Mirror images to the intermediate registry server:

        oc image mirror \
          -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping-to-registry.txt \
          --filter-by-os '.*'  \
          -a $REGISTRY_AUTH_FILE \
          --insecure  \
          --skip-multiple-scopes \
          --max-per-registry=1 \
          --continue-on-error=true
        

     2. Mirror images from the intermediate registry server to the final registry server:

        oc image mirror \
          -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping-from-registry.txt \
          --filter-by-os '.*'  \
          -a $REGISTRY_AUTH_FILE \
          --insecure  \
          --skip-multiple-scopes \
          --max-per-registry=1 \
          --continue-on-error=true
        

        The oc image mirror --help command can be run to see all the options available on the mirror command. Note that we use continue-on-error to indicate that the command should try to mirror as much as possible and continue on errors.

        oc image mirror --help
        

        Note: Sometimes based on the number and size of images to be mirrored, the oc image mirror might take longer. If you are issuing the command on a remote machine it is recommended that you run the command in the background with a nohup so even if network connection to your remote machine is lost or you close the terminal the mirroring will continue. For example, the below command will start the mirroring process in background and write the log to my-mirror-progress.txt.

        nohup oc image mirror \
        -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping.txt \
        -a $REGISTRY_AUTH_FILE \
        --filter-by-os '.*' \
        --insecure \
        --skip-multiple-scopes \
        --max-per-registry=1 \
        --continue-on-error=true > my-mirror-progress.txt  2>&1 &
        

        You can view the progress of the mirror by issuing the following command on the remote machine:

        tail -f my-mirror-progress.txt
        

   • For mirroring from a file system (disconnected mirroring):
     Mirror images to your file system:

      export IMAGE_PATH=<image-path>
      oc image mirror \
        -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping-to-filesystem.txt \
        --filter-by-os '.*'  \
        -a $REGISTRY_AUTH_FILE \
        --insecure  \
        --skip-multiple-scopes \
        --max-per-registry=1 \
        --continue-on-error=true \
        --dir "$IMAGE_PATH"
     

     The <image-path> refers to the local path to store the images. For example, in the previous section if provided file://local as input during generate mirror-manifests, then the preceding command will create a subdirectory v2/local inside directory referred by <image-path> and copy the images under it.

   The following command can be used to see all the options available on the mirror command. Note that continue-on-error is used to indicate that the command should try to mirror as much as possible and continue on errors.

   oc image mirror --help
   

   Note: Sometimes based on the number and size of images to be mirrored, the oc image mirror might take longer. If you are issuing the command on a remote machine, it is recommended that you run the command in the background with nohup so that even if you lose network connection to your remote machine or you close the terminal, the mirroring will continue. For example, the following command will start the mirroring process in the background and write the log to my-mirror-progress.txt.

    export IMAGE_PATH=<image-path>
    nohup oc image mirror \
      -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping-to-filesystem.txt \
      --filter-by-os '.*' \
      -a $REGISTRY_AUTH_FILE \
      --insecure \
      --skip-multiple-scopes \
      --max-per-registry=1 \
      --continue-on-error=true \
      --dir "$IMAGE_PATH" > my-mirror-progress.txt  2>&1 &
   

   You can view the progress of the mirror by issuing the following command on the remote machine:

   tail -f my-mirror-progress.txt
   

2. For disconnected mirroring only: Continue to move the following items to your file system:
   • The <image-path> directory you specified in the previous step
   • The auth file referred by $REGISTRY_AUTH_FILE
   • ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping-from-filesystem.txt
3. For disconnected mirroring only: Mirror images to the target registry from file system.

   Complete the steps in this section on your file system to copy the images from the file system to the $TARGET_REGISTRY. Your file system must be connected to the target docker registry.

   Important: If you used the placeholder value of TARGET_REGISTRY as a parameter to --final-registry at the time of generating mirror manifests, then before running the following command, find and replace the placeholder value of TARGET_REGISTRY in the file, images-mapping-from-filesystem.txt, with the actual registry where you want to mirror the images. For example, if you want to mirror images to myregistry.com/mynamespace then replace TARGET_REGISTRY with myregistry.com/mynamespace.
   1. Run the following command to copy the images (referred in the images-mapping-from-filesystem.txt file) from the directory referred by <image-path> to the final target registry:

      export IMAGE_PATH=<image-path>
      oc image mirror \
        -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping-from-filesystem.txt \
        -a $REGISTRY_AUTH_FILE \
        --from-dir "$IMAGE_PATH" \
        --filter-by-os '.*' \
        --insecure \
        --skip-multiple-scopes \
        --max-per-registry=1 \
        --continue-on-error=true

Configure the cluster

1. Update the global image pull secret for your Red Hat OpenShift cluster. Follow the steps in Updating the global cluster pull secret.

   The documented steps in the link enable your cluster to have proper authentication credentials in place to pull images from your TARGET_REGISTRY as specified in the image-content-source-policy.yaml which you will apply to your cluster in the next step.

2. Create ImageContentSourcePolicy
   Important:

   • Before you run the command in this step, you must be logged into your OpenShift cluster. Using the oc login command, log in to the Red Hat OpenShift Container Platform cluster where your final location resides. You can identify your specific oc login by clicking the user drop-down menu in the Red Hat OpenShift Container Platform console, then clicking Copy Login Command.
     • If you used the placeholder value of TARGET_REGISTRY as a parameter to --final-registry at the time of generating mirror manifests, then before running the following command, find and replace the placeholder value of TARGET_REGISTRY in file, ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/image-content-source-policy.yaml with the actual registry where you want to mirror the images. For example, replace TARGET_REGISTRY with myregistry.com/mynamespace.

   Run the following command to create ImageContentSourcePolicy:

   oc apply -f  ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/image-content-source-policy.yaml
   

   If you are using Red Hat OpenShift Container Platform version 4.7 or earlier, this step might cause your cluster nodes to drain and restart sequentially to apply the configuration changes.

3. Verify that the ImageContentSourcePolicy resource is created.

   oc get imageContentSourcePolicy
   

4. Verify your cluster node status and wait for all the nodes to be restarted before proceeding.

   oc get MachineConfigPool
   

   $ oc get MachineConfigPool -w
   NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
   master   rendered-master-53bda7041038b8007b038c08014626dc   True      False      False      3              3                   3                     0                      10d
   worker   rendered-worker-b54afa4063414a9038958c766e8109f7   True      False      False      3              3                   3                     0                      10d
   

   After the ImageContentsourcePolicy and global image pull secret are applied, the configuration of your nodes will be updated sequentially. Wait until all MachineConfigPools are in the UPDATED=True status before proceeding.

5. Go to the project where deployment has to be done:
   Note: You must be logged into a cluster before performing the following steps.

   export NAMESPACE=<YOUR_NAMESPACE>

   oc new-project $NAMESPACE

6. Optional: If you use an insecure registry, you must add the target registry to the cluster insecureRegistries list.

   oc patch image.config.openshift.io/cluster --type=merge \
   -p '{"spec":{"registrySources":{"insecureRegistries":["'${TARGET_REGISTRY}'"]}}}'

7. Verify your cluster node status and wait for all the nodes to be restarted before proceeding.

   oc get MachineConfigPool -w

   After the ImageContentsourcePolicy and global image pull secret are applied, the configuration of your nodes will be updated sequentially. Wait until all MachineConfigPools are updated.

   At this point your cluster is ready for IBM Connect:Direct for UNIX deployment. The helm chart is present in ~/.ibm-pak/data/cases/$CASE_NAME/$CASE_VERSION/charts/ibm-connect-direct-1.2.x.tgz directory. Use it for deployment. Copy it in current directory.

   cp ~/.ibm-pak/data/cases/$CASE_NAME/$CASE_VERSION/charts/ibm-connect-direct-1.2.x.tgz .

   Note: Replace with version information in above command.
8. Configuration required in Helm chart: To use the image mirroring in OpenShift cluster, helm chart should be configured to use the digest value for referring to container image. Set image.digest.enabled to true in values.yaml file or pass this parameter using Helm CLI.

Setting up a repeatable mirroring process

Once you complete a CASE save, you can mirror the CASE as many times as you want to. This approach allows you to mirror a specific version of the IBM Cloud Pak into development, test, and production stages using a private container registry.

Follow the steps in this section if you want to save the CASE to multiple registries (per environment) once and be able to run the CASE in the future without repeating the CASE save process.

1. Run the following command to save the CASE to ~/.ibm-pak/data/cases/$CASE_NAME/$CASE_VERSION which can be used as an input during the mirror manifest generation:

   oc ibm-pak get \
   $CASE_NAME \
   --version $CASE_VERSION

2. Run the oc ibm-pak generate mirror-manifests command to generate the image-mapping.txt:

   oc ibm-pak generate mirror-manifests \
   $CASE_NAME \
   $TARGET_REGISTRY \
   --version $CASE_VERSION

   Then add the image-mapping.txt to the oc image mirror command:

   oc image mirror \
     -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping.txt \
     --filter-by-os '.*'  \
     -a $REGISTRY_AUTH_FILE \
     --insecure  \
     --skip-multiple-scopes \
     --max-per-registry=1 \
     --continue-on-error=true

If you want to make this repeatable across environments, you can reuse the same saved CASE cache (~/.ibm-pak/$CASE_NAME/$CASE_VERSION) instead of executing a CASE save again in other environments. You do not have to worry about updated versions of dependencies being brought into the saved cache.