Mirroring images using an intermediary container registry

If your client workstation cannot connect to the internet and to the private container registry, you must mirror images to an intermediary container registry before you can mirror the images to your private container registry.

If your client workstation can connect to the internet and to the private container registry, refer to Mirroring images directly to the private container registry.

Who needs to complete this task?: A registry administrator or a user with permissions to push images to the private container registry must complete this task.

When do you need to complete this task?

If you want to pull images from a private container registry, then complete the following this task before you install Concert.

With careful planning, identify all the components that you plan to install on the cluster to complete the task once. It is a one-time setup.
If you decide to install additional services and the images are not in your private container registry, you might need to complete this task multiple times. Repeat this as needed.

Before you begin

Before you mirror the images to the private container registry, make sure you have sufficient storage space available for the images that you plan to mirror.

About this task

Use the ibm-concert-manage command to:

Mirror the images from the IBM Entitled Registry to an intermediary container registry on the client workstation.
Mirror the images from the intermediary container registry to the private container registry.

The ibm-concert-manage mirror-images command automatically sets up an intermediary container registry on the client workstation. You can use the 127.0.0.1:12443 address for the intermediary container registry.

You must be able to move the intermediary container registry behind your firewall.

Procedure

Step 1: Use the client workstation that can connect to the internet and do the following:

export IBM_ENTITLEMENT_KEY=<YOUR IBM_ENTITLEMENT_KEY>
./ibm-concert-manage.sh login-entitled-registry

Confirm that you have access to the images that you want to mirror from the IBM Entitled Registry:
1. Inspect the IBM Entitled Registry:
  Tip: If you want to validate that you have access to the images for a specific component, you can run the following command before you run the list-images command:
```
export COMPONENTS=zen,concert,cpfs,ibm-licensing,ibm-cert-manager
```
  The ./ibm-concert-manage.sh list-images command downloads the CASE packages for the specified components. By default, the packages are downloaded from github.com or IBM. If you cannot access GitHub, then add the --from_oci=true option in the command to download the packages from the IBM Entitled Registry.
```
export COMPONENTS=zen,concert,cpfs,ibm-licensing,ibm-cert-manager
export RELEASE=5.0.1
./ibm-concert-manage.sh  list-images
```
  The output will be saved to the list_images.csv file in work/offline/${VERSION} directory.
2. Check the output for errors using the following command:
```
grep "level=fatal" list_images.csv
```
The command return images that failed because of authorization errors or network errors.
Mirror the images to the intermediary container registry.
The command automatically sets up an intermediary container registry on the client workstation. You can use the 127.0.0.1:12443 address for the intermediary container registry.
Note: This command mirrors the images for all of the components that are specified in the ${COMPONENTS} environment variable. If you want to mirror images for a specific component, you can run export COMPONENTS=<component-ID> before you run the command.
```
export PRIVATE_REGISTRY_LOCATION=127.0.0.1:12443
export COMPONENTS=zen,concert,cpfs,ibm-licensing,ibm-cert-manager
export RELEASE=5.0.1
./ibm-concert-manage.sh mirror-images
```
For each component, the command generates a log file in the work directory.
Tip: Run the following command to print out any errors in the log files:
```
grep "error" mirror_*.log
```
Make sure that the images are mirrored to the intermediary container registry:
1. Check the content available in intermediary container registry:
```
./ibm-concert-manage.sh list-images-mirrored
```
  The output will be saved to the list_images.csv file in work/offline/${VERSION} directory.
  If the list-images-mirrored throws error, then you need to list all the images from the intermediary registry using curl commands. Run the following command from ibm-aaf-utils container
```
curl -s -k https://127.0.0.1:12443/v2/_catalog?n=999
```
2. Check the output for errors using the following command:
```
grep "level=fatal" list_images.csv
```

Step 2: Move the intermediary container registry behind the firewall.

Refer the following table for options for moving the intermediary container registry.

Table 1. Options for moving the intermediary container registry
Option	Details
Use a portable compute device, such as a laptop, that you can move behind your firewall.	You can use the same device to: Mirror images from the IBM Entitled Registry to the intermediary container registry. Mirror images from the intermediary container registry to the private container registry.
Use a portable storage device, such as a USB drive, that you can move behind your firewall.	You must set up two client workstations: A workstation that can connect to the internet. From this workstation, you can mirror the images from the IBM Entitled Registry to the intermediary container registry on the portable storage device. A workstation that can connect to the private container registry. After you move the portable storage device to this workstation, you can mirror the images from the intermediary container registry to the private container registry. Important: The workstation that can connect to the private container registry must have: A copy of the `ibm-concert-manage.sh` script A copy of the work directory, which contains the CASE packages and intermediary container registry, i.e., `$WORK_DIR`
Use a file transfer protocol, such as scp or sftp to move images behind your firewall.	You must set up two client workstations: A workstation that can connect to the internet. From this workstation, you can mirror the images from the IBM Entitled Registry to the intermediary container registry. A workstation that can connect to the private container registry. After you transfer the intermediary container registry to this workstation, you can mirror the images from the intermediary container registry to the private container registry. Important: The workstation that can connect to the private container registry must have: A copy of the ibm-concert-manage.sh A copy of the work directory, which contains the CASE packages and intermediary container registry, i.e., `$WORK_DIR`

Step 3: Copy the manage script to client workstation and run ./ibm-concert-manage.sh initialize from a client workstation that connect to private container registry:

The following command assumes that you are using private container registry that is secured with credentials:

export PRIVATE_REGISTRY_LOCATION=
export PRIVATE_REGISTRY_PUSH_USER=
export PRIVATE_REGISTRY_PUSH_PASSWORD=
./ibm-concert-manage.sh login-private-registry

If your private registry is not secured, then remove the following sensitive variables:
- ${PRIVATE_REGISTRY_PUSH_USER}
- ${PRIVATE_REGISTRY_PUSH_PASSWORD}
Mirror the images from intermediary container registry to the private container registry.
Tip: The following command mirrors the images for all components specified in the ${COMPONENTS} environment variable.
```
export PRIVATE_REGISTRY_LOCATION=

./ibm-concert-manage.sh mirror-images-from-intermediate
```
A log file gets generated in the work directory for each component.
Make sure that the images are mirrored to the intermediary container registry:
1. Check the content available in intermediary container registry:
```
./ibm-concert-manage.sh list-images
```
  The output will be saved to the list_images.csv file in work/offline/${VERSION} directory.
  If the list-images-mirrored throws error, then you need to list all the images from the intermediary registry using curl commands. Run the following command from ibm-aaf-utils container
```
curl -s -k https://127.0.0.1:12443/v2/_catalog?n=999
```
2. Check the output for errors using the following command:
```
grep "level=fatal" list_images.csv
```
  The command returns the images that are missing or cannot be inspected.

Result

The images for your architecture are mirrored to the private container registry.

Note: Some components, such as the CPFS, provides only multi-arch images. For components with multi-arch images, all images are mirrored to the private container registry.

What to do next

Now that you've mirrored the images to the private container registry, you are ready to Configuring an image content source policy.

To install Concert for air gap environment, refer to Installing Concert in an air gap environment.