Mirroring catalogs to a private registry with a bastion server
You can use a bastion host to mirror the catalogs to a private registry for an airgap deployment.
About this task
The bastion host must be connected to both the private registry and the Red Hat OpenShift
Container Platform cluster. When you mirror the images, you must run the oc mirror
commands on the bastion host. The IBM Catalog Management plug-in or ibm-pak
uses
File-based catalogs (FBC) to mirror all the catalogs. The FBC is in JSON
or
YAML
format that contains information that is needed to install an operator into
the OpenShift Container Platform.
Procedure
- Generate the required mirror manifests.
- Define all the required environment variables by running the following
command:
where, <target-registry> refers to the registry where the images are mirrored to and accessed by the OpenShift Container Platform cluster. For example, you can specifyexport CASE_NAME=ibm-cp-fncm-case export CASE_VERSION=5.6.0 export IBMPAK_HOME=<path> export TARGET_REGISTRY=<target-registry> export REGISTRY_AUTH_FILE=$IBMPAK_HOME/.ibm-pak/auth.json export NAMESPACE=<namespace> export CASE_INVENTORY_SETUP=fncmOperatorSetup
localhost.localdomain:5000
or172.16.0.10:5000
as value for <target-registry>.Note: Releases with interim fixes are packaged in archives with a new minor version. The version numbers follow therelease.major.minor
standard. For example, the first interim fix for 5.6.0 is packaged in the archive ibm-cp-fncm-case-5.6.1.- (Option 1) Podman login
- If you installed Podman only on the bastion host, the authentication file is located in
$XDG_RUNTIME_DIR/containers/auth.json. Run the following command to define the
REGISTRY_AUTH_FILE
environment variable to point to the authentication file location.
After you define theexport REGISTRY_AUTH_FILE=$XDG_RUNTIME_DIR/containers/auth.json
REGISTRY_AUTH_FILE
environment variable, run thepodman login
command to see that the file is populated with registry credentials. - (Option 2) Docker login
- If you installed Docker on the bastion host, the authentication file is located in
$HOME/.docker/config.json on Linux® or
%USERPROFILE%/.docker/config.json on Windows. After you run the
docker login
command, you can exportREGISTRY_AUTH_FILE
to point to that location. For example, on Linux you can run the following command:export REGISTRY_AUTH_FILE=$HOME/.docker/config.json
- Run the following command to set the preferred tool parameter in the YAML file to
oc-mirror
.oc ibm-pak config mirror-tools --enabled oc-mirror
- Run the following command to generate mirror manifests to be used when mirroring the catalog to
the target registry. The $TARGET_REGISTRY refers to the registry where the
catalogs are mirrored to and accessed by the OpenShift Container Platform
cluster.
oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY \ --version $CASE_VERSION
Important: Thegenerate mirror-manifests
command provides an output that lists the command for mirroring. Save the command for later use.The following example shows sample output.oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY --version $CASE_VERSION ibm-cp-fncm-case-5.6.0 ∙∙● 18s running catalog based merge catalog validation... Generating Catalog ∙●∙ 15s Generating mirror manifests of CASE: ibm-cp-fncm-case, version: 5.6.0 is complete Next steps - To mirror the non curated catalog: oc mirror --config $IBMPAK_HOME/.ibm-pak/data/mirror/ibm-cp-fncm-case/$CASE_VERSION/image-set-config.yaml docker://$TARGET_REGISTRY --dest-skip-tls --max-per-registry=6 Note: oc mirror arguments are recommendations and may not be valid on all versions of the oc-mirror plug-in.
Note: After an upgrade, you can remove earlier version images from the local registry by including an alternative namespace to copy the images to instead of the default path. For example, the following CASE command copies the images to alocationX
namespace.oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY/locationX \ --version $CASE_VERSION
A new directory $IBMPAK_HOME/.ibm-pak/mirror is created when you issue the
oc ibm-pak generate mirror-manifests
command. The mirror directory stores the catalog-sources.yaml, image-content-source-policy.yaml, and image-set-config.yaml files.Tip: If you are using a Red Hat®Quay.io
registry and need to mirror the images to a specific organization in the registry, you can set the target to that organization. Specify the organization name in thegenerate mirror-manifests
command:export ORGANIZATION=<your-organization> \ oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY/$ORGANIZATION \ --version $CASE_VERSION
Restriction: Currently, you cannot select the images to mirror by their target architecture because image registries do not support sparse manifests (manifests that reference image digests outside of the package).The $IBMPAK_HOME/.ibm-pak directory structure is built over time as you save CASE files and mirror them. The following tree shows an example of the $IBMPAK_HOME/.ibm-pak directory structure:
tree $IBMPAK_HOME/.ibm-pak $IBMPAK_HOME/.ibm-pak ├── config │ └── config.yaml ├── data │ ├── cases │ │ └── ibm-cp-fncm-case │ │ └── 5.6.x │ │ ├── XXXXX │ │ ├── XXXXX │ └── mirror │ └── ibm-cp-fncm-case │ └── 5.6.x │ ├── catalog-sources.yaml │ ├── image-content-source-policy.yaml │ └── image-set-config.yaml └── logs └── oc-ibm_pak.log
- Define all the required environment variables by running the following
command:
-
Authenticate the registries.
You must store authentication credentials for all the FileNet Content Manager source Docker registries. The following registries require authentication:
cp.icr.io
- <target-registry> where the images are mirrored to and accessed by the OpenShift Container Platform cluster
Run the following command to configure credentials for all target registries that require authentication. Run the command separately for each registry.
podman login cp.icr.io podman login <TARGET_REGISTRY>
Important: When you log in tocp.icr.io
, you must specify the user ascp
and the IBM entitlement key as the password. For example:podman login cp.icr.io Username: cp Password: xxxxxxxxxxxxxxxxxxxxx Login Succeeded!
The password can be copied from the IBM container library. If you see "cert error" messages, you can add
--tls-verify=false
to the command. -
Run the command generated in step 1c to mirror the catalogs.
Important: For new installations, by default, the
oc mirror
command mirrors all the images from all channels in the FNCM catalog. You can edit the $IBMPAK_HOME/.ibm-pak/data/publish/latest/image-set-config.yaml file to remove the OLM channels and keep only the v24.0 channel.The following is an example command.oc mirror --config $IBMPAK_HOME/.ibm-pak/data/mirror/ibm-cp-fncm-case/$CASE_VERSION/image-set-config.yaml docker://$TARGET_REGISTRY --dest-skip-tls --max-per-registry=6
Run the command on the host that is connected to both the local Docker registry and the OpenShift Container Platform cluster.
The command does not produce any console logs for about 6 - 8 minutes as it prepares the list from the CASE package. If you want, you can add verbose (-v) to the command with possible values of 1 to 9.
The following command can be used to see all the available options.oc mirror --help
Based on the number and size of the images to mirror, theoc mirror
command can take a considerable amount of time. If you are running the command on a remote system, run the command in the background with thenohup
POSIX command so that it does not stop if the user logs out. The following command starts the mirroring process in the background and writes the log to a fncm-560.txt file.nohup oc mirror --config $IBMPAK_HOME/.ibm-pak/data/mirror/ibm-cp-fncm-case/$CASE_VERSION/image-set-config.yaml docker://$TARGET_REGISTRY --dest-skip-tls --max-per-registry=6 > /opt/fncm-560.txt 2>&1 &
The following example shows sample output.
nohup: ignoring input Checking push permissions for $TARGET_REGISTRY Creating directory: oc-mirror-workspace/src/publish Creating directory: oc-mirror-workspace/src/v2 Creating directory: oc-mirror-workspace/src/charts Creating directory: oc-mirror-workspace/src/release-signatures No metadata detected, creating new workspace 32 related images processed in 566.390701ms Writing image mapping to oc-mirror-workspace/operators.1711048843/manifests-manifest-list/mapping.txt -- -- -- info: Mirroring completed in 320ms (0B/s) Rendering catalog image "$TARGET_REGISTRY/ibm-catalog:5d0b72" with file-based catalog Writing image mapping to oc-mirror-workspace/results-1711048875/mapping.txt Writing CatalogSource manifests to oc-mirror-workspace/results-1711048875 Writing ICSP manifests to oc-mirror-workspace/results-1711048875
- Update the global image pull secret for your Red Hat OpenShift cluster to have authentication credentials in place to pull images from your $TARGET_REGISTRY as specified in the image-content-source-policy.yaml file. For more information, see Updating the global cluster pull secret.
- Run the following command to create
ImageContentsourcePolicy
.oc apply -f $IBMPAK_HOME/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/image-content-source-policy.yaml
- Verify that the
ImageContentsourcePolicy
resource is created.oc get imageContentSourcePolicy
- Verify your cluster node status.
oc get MachineConfigPool -w
After the
ImageContentsourcePolicy
and global image pull secret are applied, the configuration of your nodes is updated sequentially. Wait until all theMachineConfigPools
are updated before you move on to the next step. -
Create a project for the CASE commands (
fncm
is an example) by running the following commands. Before you run the command in this step, you must be logged in to your OpenShift cluster.oc new-project $NAMESPACE
- Optional: If you use an insecure registry, you must add the target registry to the
cluster
insecureRegistries
list.oc patch image.config.openshift.io/cluster \ --type=merge -p '{"spec":{"registrySources":{"insecureRegistries":["'${TARGET_REGISTRY}'"]}}}'
What to do next
You can now go ahead and install the FileNet Content Manager catalog and operator. For more information, see Installing the FileNet Content Manager catalog and an operator instance in an air gap environment.