Mirroring catalogs to a private registry with a bastion server

You can use a bastion host to mirror the catalogs to a private registry for an airgap deployment.

About this task

The bastion host must be connected to both the private registry and the Red Hat OpenShift Container Platform cluster. When you mirror the images, you must run the oc mirror commands on the bastion host. The IBM Catalog Management plug-in or ibm-pak uses File-based catalogs (FBC) to mirror all the catalogs. The FBC is in JSON or YAML format that contains information that is needed to install an operator into the OpenShift Container Platform.

Procedure

  1. Generate the required mirror manifests.
    1. Define all the required environment variables by running the following command:
      export CASE_NAME=ibm-cp-fncm-case
      export CASE_VERSION=5.6.0
      export IBMPAK_HOME=<path>
      export TARGET_REGISTRY=<target-registry>
      export REGISTRY_AUTH_FILE=$IBMPAK_HOME/.ibm-pak/auth.json
      export NAMESPACE=<namespace>
      export CASE_INVENTORY_SETUP=fncmOperatorSetup
      where, <target-registry> refers to the registry where the images are mirrored to and accessed by the OpenShift Container Platform cluster. For example, you can specify localhost.localdomain:5000 or 172.16.0.10:5000 as value for <target-registry>.
      Note: Releases with interim fixes are packaged in archives with a new minor version. The version numbers follow the release.major.minor standard. For example, the first interim fix for 5.6.0 is packaged in the archive ibm-cp-fncm-case-5.6.1.
      (Option 1) Podman login
      If you installed Podman only on the bastion host, the authentication file is located in $XDG_RUNTIME_DIR/containers/auth.json. Run the following command to define the REGISTRY_AUTH_FILE environment variable to point to the authentication file location.
      export REGISTRY_AUTH_FILE=$XDG_RUNTIME_DIR/containers/auth.json
      After you define the REGISTRY_AUTH_FILE environment variable, run the podman login command to see that the file is populated with registry credentials.
      (Option 2) Docker login
      If you installed Docker on the bastion host, the authentication file is located in $HOME/.docker/config.json on Linux® or %USERPROFILE%/.docker/config.json on Windows. After you run the docker login command, you can export REGISTRY_AUTH_FILE to point to that location. For example, on Linux you can run the following command:
      export REGISTRY_AUTH_FILE=$HOME/.docker/config.json
    2. Run the following command to set the preferred tool parameter in the YAML file to oc-mirror.
      oc ibm-pak config mirror-tools --enabled oc-mirror
    3. Run the following command to generate mirror manifests to be used when mirroring the catalog to the target registry. The $TARGET_REGISTRY refers to the registry where the catalogs are mirrored to and accessed by the OpenShift Container Platform cluster.
      oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY \
        --version $CASE_VERSION
      Important: The generate mirror-manifests command provides an output that lists the command for mirroring. Save the command for later use.
      The following example shows sample output.
      oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY --version $CASE_VERSION
      ibm-cp-fncm-case-5.6.0 ∙∙● 18s
      running catalog based merge catalog validation...
      Generating Catalog ∙●∙ 15s
      Generating mirror manifests of CASE: ibm-cp-fncm-case, version: 5.6.0 is complete
      
      Next steps
      
      - To mirror the non curated catalog:
      
      oc mirror --config $IBMPAK_HOME/.ibm-pak/data/mirror/ibm-cp-fncm-case/$CASE_VERSION/image-set-config.yaml docker://$TARGET_REGISTRY --dest-skip-tls --max-per-registry=6
      
      Note: oc mirror arguments are recommendations and may not be valid on all versions of the oc-mirror plug-in.
      Note: After an upgrade, you can remove earlier version images from the local registry by including an alternative namespace to copy the images to instead of the default path. For example, the following CASE command copies the images to a locationX namespace.
      oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY/locationX \
        --version $CASE_VERSION

      A new directory $IBMPAK_HOME/.ibm-pak/mirror is created when you issue the oc ibm-pak generate mirror-manifests command. The mirror directory stores the catalog-sources.yaml, image-content-source-policy.yaml, and image-set-config.yaml files.

      Tip: If you are using a Red Hat® Quay.io registry and need to mirror the images to a specific organization in the registry, you can set the target to that organization. Specify the organization name in the generate mirror-manifests command:
      export ORGANIZATION=<your-organization> \
      oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY/$ORGANIZATION \
         --version $CASE_VERSION
    Restriction: Currently, you cannot select the images to mirror by their target architecture because image registries do not support sparse manifests (manifests that reference image digests outside of the package).

    The $IBMPAK_HOME/.ibm-pak directory structure is built over time as you save CASE files and mirror them. The following tree shows an example of the $IBMPAK_HOME/.ibm-pak directory structure:

    tree $IBMPAK_HOME/.ibm-pak
    $IBMPAK_HOME/.ibm-pak
    ├── config
    │   └── config.yaml
    ├── data
    │   ├── cases
    │   │   └── ibm-cp-fncm-case
    │   │       └── 5.6.x
    │   │           ├── XXXXX
    │   │           ├── XXXXX
    │   └── mirror
    │       └── ibm-cp-fncm-case
    │           └── 5.6.x
    │               ├── catalog-sources.yaml
    │               ├── image-content-source-policy.yaml
    │               └── image-set-config.yaml
    └── logs
        └── oc-ibm_pak.log
  2. Authenticate the registries.

    You must store authentication credentials for all the FileNet Content Manager source Docker registries. The following registries require authentication:

    • cp.icr.io
    • <target-registry> where the images are mirrored to and accessed by the OpenShift Container Platform cluster

    Run the following command to configure credentials for all target registries that require authentication. Run the command separately for each registry.

    podman login cp.icr.io
    podman login <TARGET_REGISTRY>
    Important: When you log in to cp.icr.io, you must specify the user as cp and the IBM entitlement key as the password. For example:
    podman login cp.icr.io
    Username: cp
    Password: xxxxxxxxxxxxxxxxxxxxx
    Login Succeeded!
    

    The password can be copied from the IBM container library. If you see "cert error" messages, you can add --tls-verify=false to the command.

  3. Run the command generated in step 1c to mirror the catalogs.
    Important: For new installations, by default, the oc mirror command mirrors all the images from all channels in the FNCM catalog. You can edit the $IBMPAK_HOME/.ibm-pak/data/publish/latest/image-set-config.yaml file to remove the OLM channels and keep only the v24.0 channel.
    The following is an example command.
    oc mirror --config $IBMPAK_HOME/.ibm-pak/data/mirror/ibm-cp-fncm-case/$CASE_VERSION/image-set-config.yaml docker://$TARGET_REGISTRY --dest-skip-tls --max-per-registry=6

    Run the command on the host that is connected to both the local Docker registry and the OpenShift Container Platform cluster.

    The command does not produce any console logs for about 6 - 8 minutes as it prepares the list from the CASE package. If you want, you can add verbose (-v) to the command with possible values of 1 to 9.

    The following command can be used to see all the available options.
    oc mirror --help
    Based on the number and size of the images to mirror, the oc mirror command can take a considerable amount of time. If you are running the command on a remote system, run the command in the background with the nohup POSIX command so that it does not stop if the user logs out. The following command starts the mirroring process in the background and writes the log to a fncm-560.txt file.
    nohup oc mirror --config $IBMPAK_HOME/.ibm-pak/data/mirror/ibm-cp-fncm-case/$CASE_VERSION/image-set-config.yaml docker://$TARGET_REGISTRY --dest-skip-tls --max-per-registry=6 > /opt/fncm-560.txt 2>&1 &

    The following example shows sample output.

    nohup: ignoring input
    Checking push permissions for $TARGET_REGISTRY
    Creating directory: oc-mirror-workspace/src/publish
    Creating directory: oc-mirror-workspace/src/v2
    Creating directory: oc-mirror-workspace/src/charts
    Creating directory: oc-mirror-workspace/src/release-signatures
    No metadata detected, creating new workspace
    32 related images processed in 566.390701ms
    Writing image mapping to oc-mirror-workspace/operators.1711048843/manifests-manifest-list/mapping.txt
    --
    --
    --
    info: Mirroring completed in 320ms (0B/s)
    Rendering catalog image "$TARGET_REGISTRY/ibm-catalog:5d0b72" with file-based catalog
    Writing image mapping to oc-mirror-workspace/results-1711048875/mapping.txt
    Writing CatalogSource manifests to oc-mirror-workspace/results-1711048875
    Writing ICSP manifests to oc-mirror-workspace/results-1711048875
    
  4. Update the global image pull secret for your Red Hat OpenShift cluster to have authentication credentials in place to pull images from your $TARGET_REGISTRY as specified in the image-content-source-policy.yaml file. For more information, see Updating the global cluster pull secret.
  5. Run the following command to create ImageContentsourcePolicy.
    oc apply -f $IBMPAK_HOME/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/image-content-source-policy.yaml
  6. Verify that the ImageContentsourcePolicy resource is created.
    oc get imageContentSourcePolicy
  7. Verify your cluster node status.
    oc get MachineConfigPool -w

    After the ImageContentsourcePolicy and global image pull secret are applied, the configuration of your nodes is updated sequentially. Wait until all the MachineConfigPools are updated before you move on to the next step.

  8. Create a project for the CASE commands (fncm is an example) by running the following commands. Before you run the command in this step, you must be logged in to your OpenShift cluster.
    oc new-project $NAMESPACE
  9. Optional: If you use an insecure registry, you must add the target registry to the cluster insecureRegistries list.
    oc patch image.config.openshift.io/cluster \
      --type=merge -p '{"spec":{"registrySources":{"insecureRegistries":["'${TARGET_REGISTRY}'"]}}}' 

What to do next

You can now go ahead and install the FileNet Content Manager catalog and operator. For more information, see Installing the FileNet Content Manager catalog and an operator instance in an air gap environment.