Mirroring images to your private container registry

Guardium®External S-TAP images are accessible from the IBM Cloud® Container Registry. In many situations,Guardium recommends that you mirror the necessary software images from the IBM Cloud Container Registry to a private container registry.

Important: If your cluster is air-gapped (also called an offline or disconnected cluster), then you must mirror the necessary images to your private container registry .

Even if your environment is not air-gapped, consider using a private container registry if you want to:

Run security scans against the software images before you install them on your cluster.
Ensure that you have the same images available for multiple deployments, such as development or test environments and production environments.

IBM suggests that you pull images directly from the IBM Cloud Container Registry if your cluster is not air-gapped, your network is extremely reliable, and latency is not a concern. However, if your system does not meet all of those requirements, mirror the images to a private container registry for predictable and reliable performance.

Define the environment variables

The first step is to define the environment variables that describe your private registry. For example:

export REGISTRY_IMAGE=docker.io/library/registry:2.7
export REGISTRY_HOST=`uname -n`
export REGISTRY_PORT=5000
export REGISTRY_USERNAME="user"
export REGISTRY_PASSWORD="pass"

Create a private registry (optional)

An example of how to create an insecure registry on a bastion node follows.

Define the environment variables for the registry that you want to create. For example:

 REGISTRY_IMAGE=docker.io/library/registry:2.7 
export REGISTRY_NAME=docker-registry 
export REGISTRY_HOST=`uname -n` 
export REGISTRY_PORT=5000 
export REGISTRY_DIR=/root/registry 
export REGISTRY_TLS_CA_SUBJECT="/C=US/ST=New York/L=Armonk/O=IBM/CN=IBM CA" 
export REGISTRY_TLS_CERT_SUBJECT="/C=US/ST=New York/L=Armonk/O=IBM/CN=$REGISTRY_HOST" 
export REGISTRY_TLS_CERT_SUBJECT_ALT_NAME="subjectAltName=IP:127.0.0.1,DNS:localhost,DNS:$REGISTRY_HOST" 
export REGISTRY_USERNAME="user" 
export REGISTRY_PASSWORD="pass"

Prepare the persistent storage:

mkdir -p $REGISTRY_DIR
mkdir -p $REGISTRY_DIR/data
mkdir -p $REGISTRY_DIR/auth
mkdir -p $REGISTRY_DIR/certs
echo $REGISTRY_HOST > $REGISTRY_DIR/hostname

Configure authentication:

dnf install httpd-tools
htpasswd -bBc $REGISTRY_DIR/auth/htpasswd $REGISTRY_USERNAME $REGISTRY_PASSWORD

Generate the certificate:

openssl genrsa -out $REGISTRY_DIR/certs/ca.key 4096
openssl req -new -x509 -days 365 -sha256 -subj "${REGISTRY_TLS_CA_SUBJECT}" -key "${REGISTRY_DIR}/certs/ca.key" -out "${REGISTRY_DIR}/certs/ca.crt"
openssl req -newkey rsa:4096 -nodes -subj "${REGISTRY_TLS_CERT_SUBJECT}" -keyout "${REGISTRY_DIR}/certs/server.key" -out "${REGISTRY_DIR}/certs/server.csr"
openssl x509 -req -days 365 -sha256 -extfile <(printf "${REGISTRY_TLS_CERT_SUBJECT_ALT_NAME}") -CAcreateserial -CA "${REGISTRY_DIR}/certs/ca.crt" -CAkey "${REGISTRY_DIR}/certs/ca.key" -in "${REGISTRY_DIR}/certs/server.csr"   -out "${REGISTRY_DIR}/certs/server.crt"

Start the registry:

podman run --name "${REGISTRY_NAME}" -p ${REGISTRY_PORT}:5000 --restart=always \
            -v ${REGISTRY_DIR}/data:/var/lib/registry:z \
            -v ${REGISTRY_DIR}/auth:/auth:z \
            -v ${REGISTRY_DIR}/certs:/certs:z \
            -e REGISTRY_AUTH=htpasswd \
            -e REGISTRY_AUTH_HTPASSWD_REALM=RegistryRealm \
            -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
            -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/server.crt \
            -e REGISTRY_HTTP_TLS_KEY=/certs/server.key \
            -d ${REGISTRY_IMAGE}

Store the certificate for the registry in a configmap in the openshift-config namespace:

oc create configmap registry-cas -n openshift-config --from-file=$REGISTRY_NAME..$REGISTRY_PORT=$REGISTRY_DIR/certs/server.crt

Add the new configmap to the trust store for the OpenShift cluster:

oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"registry-cas"}}}' --type=merge

Cloud Pak for Data images

IBM Cloud Pak software uses the following prefixes to identify images. These publicly available images are provided by IBM. The images do not require an entitlement key to download.

cp.icr.io/cpopen - IBM Cloud Pak® for Data images.
icr.io/guardium-insight - IBM® Guardium Insights images.

Ensure that:

Your private container registry is configured to allow these prefixes.
The credentials that you use to push images to the private container registry can push images with these prefixes.

Methods for mirroring images

There are several ways that you can mirror images from the IBM Cloud Container Registry to your private container registry. Choose the most appropriate method for your environment:

Method	Description	Connected clusters	Air-gapped clusters
Portable compute device	Example: A laptop that you can move behind your firewall is a portable compute device. High-level process using a portable compute device: Create an intermediary container registry on a portable compute device that is connected to the internet. From the portable compute device, mirror images from the IBM Cloud Container Registry to the intermediary container registry. Bring the device behind your firewall and mirror the images from the intermediary container registry to the private container registry that is accessible from the Red Hat OpenShift Container Platform cluster.		✓
File transfer	Example: You can either use a portable storage device, such as a USB drive, or use `scp` or `sftp` to move images behind your firewall. High-level process using a file transfer: Create an intermediary container registry. If you are using a portable storage device, create the intermediary container registry on the storage device. From a workstation that can connect to the internet and the intermediary container registry, mirror the images from the IBM Cloud Container Registry to the intermediary container registry. Move the files and or the storage device behind your firewall. Set up a workstation behind the firewall to mirror the images to the private container registry that is accessible from the Red Hat OpenShift Container Platform cluster.		✓
Bastion node	Example: A server with access to both the public internet and the private container registry that is accessible from the Red Hat OpenShift Container Platform cluster. If you use a bastion node, replicate the images from the IBM Cloud Container Registry to the private container registry that is accessible from the Red Hat OpenShift Container Platform cluster.	✓	✓

Pull and mirror images

Note: Best practice: You can run the commands in this task exactly as written if you set up environment variables that are described in step 2 of Deploying External S-TAP with an operator. Run the environment variable script before you run the commands in this task.

Take the following steps to mirror the images to a private registry.

Generate the mirror image list from a machine that can access the IBM Cloud Registry (icr.io):

pushd $WORKDIR; oc ibm-pak generate mirror-manifests $CASE_NAME MIRROR --version $CASE_VERSION; popd

Download the images:

Create a directory to store the mirrored images:
```
mkdir -p $WORKDIR/.ibm-pak/data/mirror/export
```

Download the images to the local directory:

pushd $WORKDIR ; while read MAPPING; do SOURCE="docker://`echo $MAPPING | cut -d'=' -f1`";
 DEST="dir:$WORKDIR/.ibm-pak/data/mirror/export/`echo $MAPPING | cut -d'=' -f2 | sed "s/^.*\///"`"; echo "Copying $SOURCE -> $DEST" ; 
 skopeo copy $SOURCE $DEST --remove-signatures --src-tls-verify=false --dest-tls-verify=false ;
 done < $WORKDIR/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping.txt ; popd

From a system that has access to the private container registry, take the following steps:

Copy the .ibm-pak directory to a machine with access to the internal registry, then continue to the next step on that machine.

Push the images to the private repository:

pushd $WORKDIR ; while read MAPPING; do DEST="docker://$REGISTRY_HOST:$REGISTRY_PORT/`echo $MAPPING | cut -d'=' -f2 | sed "s/^[^/]*\///"`";
 SOURCE="dir:$WORKDIR/.ibm-pak/data/mirror//export/`echo $MAPPING | cut -d'=' -f2 | sed "s/^.*\///"`"; echo "Copying $SOURCE -> $DEST" ;
 skopeo copy $SOURCE $DEST --remove-signatures --src-tls-verify=false --dest-tls-verify=false --dest-creds
 "$REGISTRY_USERNAME:$REGISTRY_PASSWORD" ; done < $WORKDIR/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping.txt ; popd

What's next

After you create the container and store it in the registry, you need to configure the global pull secret, as described in Configuring the cluster to pull images.