Pre-upgrade preparation for air-gapped environment using a bastion host

In a disconnected, network-restricted environment, prepare your API Connect upgrade sources on your bastion host.

Before you begin

Review and complete all steps that are documented in Planning your API Connect upgrade on OpenShift and Pre-upgrade preparation and checks on OpenShift.
The upgrade procedure requires you to use Red Hat Skopeo for moving container images. Skopeo is not available for Microsoft Windows, so you cannot perform this task with a Windows host.

Attention: Some steps in this task use tilde character: ~. Do not use this character within double quotation marks in any command because the tilde doesn’t expand and the command might fail.

About this task

If you previously deployed or upgraded API Connect in an air-gapped environment, you can use the same bastion host as before.

Procedure

Set up the mirroring environment.

Prepare the target cluster:
- Deploy a supported version of Red Hat OpenShift Container Platform (OCP) as a cluster.
  For information, see Table 2 "API Connect and OpenShift Container Platform (OCP) compatibility matrix" in IBM API Connect Version 10 software product compatibility requirements.
- Configure storage on the cluster and make sure that it is available.

Prepare the bastion host:

You must be able to connect your bastion host to the internet and to the restricted network environment (with access to the Red Hat OpenShift Container Platform (OCP) cluster and the local registry) at the same time. Your host must be on a Linux x86_64 or Mac platform with any operating system that the Red Hat OpenShift Client supports (in Windows, execute the actions in a Linux x86_64 VM or from a Windows Subsystem for Linux terminal).

Ensure that the sites and ports listed in Table 1 can be reached from the bastion host:

Table 1. Sites that must be reached from the bastion host
Site	Description
`icr.io:443`	IBM entitled registry
`quay.io:443`	Local API Connect image repository
`github.com`	CASE files and tools
`redhat.com`	Red Hat OpenShift Container Platform (OCP) upgrades

On the bastion host, install either Docker or Podman (not both).
Docker and Podman are used for managing containers; you only need to install one of these applications.
- To install Docker (for example, on Red Hat Enterprise Linux), run the following commands:
```
yum check-update
yum install docker
```
- To install Podman, see the Podman installation instructions.
  For example, on Red Hat Enterprise Linux 9, install Podman with the following command:
```
yum install podman
```
Install the Red Hat OpenShift Client tool (oc) as explained in Getting started with the OpenShift CLI.
The oc tool is used for managing Red Hat OpenShift resources in the cluster.
Download the IBM Catalog Management Plug-in for IBM Cloud Paks version 1.1.0 or later from GitHub.
The ibm-pak plug-in enables you to access hosted product images, and to run oc ibm-pak commands against the cluster. To confirm that ibm-pak is installed, run the following command and verify that the response lists the command usage:
```
oc ibm-pak --help
```

Set up a local image registry and credentials.
The local Docker registry stores the mirrored images in your network-restricted environment.
1. Install a registry, or get access to an existing registry.
  You might already have access to one or more centralized, corporate registry servers to store the API Connect images. If not, then you must install and configure a production-grade registry before proceeding.
  The registry product that you use must meet the following requirements:
  - Supports multi-architecture images through Docker Manifest V2, Schema 2
    For details, see Docker Manifest V2, Schema 2.
  - Is accessible from the Red Hat OpenShift Container Platform cluster nodes
  - Allows path separators in the image name
  Note: Do not use the Red Hat OpenShift image registry as your local registry because it does not support multi-architecture images or path separators in the image name.
2. Configure the registry to meet the following requirements:
  - Supports auto-repository creation
  - Has sufficient storage to hold all of the software that is to be transferred
  - Has the credentials of a user who can create and write to repositories (the mirroring process uses these credentials)
  - Has the credentials of a user who can read all repositories (the Red Hat OpenShift Container Platform cluster uses these credentials)
To access your registries during an air-gapped installation, use an account that can write to the target local registry. To access your registries during runtime, use an account that can read from the target local registry.

Mirror the cert-manager operator for Red Hat OpenShift.
Mirror the operator as explained in the Red Hat instructions: Mirroring images for a disconnected installation using the oc-mirror plugin .
When you modify the ImageSetConfiguration, include openshift-cert-manager-operator at channel stable-v1 as one of the packages, as shown in the following example:
```
    packages:
    - name: openshift-cert-manager-operator
      channels:
      - name: stable-v1
```
Set environment variables and download the CASE files.

Create environment variables to use while mirroring images, connect to the internet, and download the CASE files needed for installing API Connect.
1. Create the following environment variables with the installer image name and the image inventory on your host:
  - API Connect:
```
export CASE_NAME=ibm-apiconnect
export CASE_VERSION=5.3.0
export ARCH=amd64
```
    For information about API Connect CASE versions and their corresponding operators and operands, see Operator, CASE, and platform requirements.
  - Foundational services:
```
export CS_CASE_NAME=ibm-cp-common-services
export CS_CASE_VERSION=4.6.3
export CS_ARCH=amd64
```
    For information about IBM Cloud Pak foundational services (common services) CASE versions, see "Table 1. Image versions for offline installation" in Installing IBM Cloud Pak foundational services in an air-gapped environment in the IBM Cloud Pak foundational services documentation.
  - EDB:
```
export EDB_CASE_NAME=ibm-cloud-native-postgresql
export EDB_CASE_VERSION=5.2.0
```
2. Connect your host to the internet (it does not need to be connected to the network-restricted environment yet).
3. Download the CASE files to your host:
  - API Connect:
```
oc ibm-pak get $CASE_NAME --version $CASE_VERSION
```
  - Foundational services:
```
oc ibm-pak get $CS_CASE_NAME --version $CS_CASE_VERSION
```
  - EDB:
```
oc ibm-pak get $EDB_CASE_NAME --version $EDB_CASE_VERSION
```
Generate mirror manifests.

The process of mirroring images pulls the images from the internet and pushes them to your local registry. After mirroring your images, you can configure your cluster and pull the images to it before you install API Connect.
1. Define the environment variable $TARGET_REGISTRY by running the following command:
```
export TARGET_REGISTRY=<target-registry>
```
  Replace <target-registry> with the IP address (or hostname) and port of the local registry; for example: 172.16.0.10:5000. If you want the images to use a specific namespace within the target registry, you can specify it here; for example: 172.16.0.10:5000/registry_ns.
2. Generate mirror manifests by running the following commands:
  - API Connect:
```
oc ibm-pak generate mirror-manifests $CASE_NAME --version $CASE_VERSION $TARGET_REGISTRY
```
  - Foundational services:
```
oc ibm-pak generate mirror-manifests $CS_CASE_NAME --version $CS_CASE_VERSION $TARGET_REGISTRY
```
  - EDB:
```
oc ibm-pak generate mirror-manifests $EDB_CASE_NAME --version $EDB_CASE_VERSION $TARGET_REGISTRY
```
  If you need to filter for a specific image group, add the parameter --filter <image_group> to the command.
  The generate command creates the following files at ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION, ~/.ibm-pak/data/mirror/$CS_CASE_NAME/$CS_CASE_VERSION, and ~/.ibm-pak/data/mirror/$EDB_CASE_NAME/$EDB_CASE_VERSION:
  - catalog-sources.yaml
  - catalog-sources-linux-<arch>.yaml (if there are architecture-specific catalog sources)
  - image-content-source-policy.yaml
  - images-mapping.txt
  The files are used when mirroring the images to the TARGET_REGISTRY.
Authenticate with the entitled registry where the images are hosted.
1. Obtain an entitlement key for the registry where the images are hosted:
  1. Log in to the IBM Container Library.
  2. In the Container software library, select Get entitlement key.
  3. In the "Access your container software" section, click Copy key.
  4. Copy the key to a safe location; you will use it to log in to cp.icr.io in the next step.
2. Login to cp.icr.io registry:
  The image pull secret is used to authenticate with the registry and access product images.
  1. Run the following command to export the path to the file that stores the authentication credentials that are generated on a Podman or Docker login:
```
export REGISTRY_AUTH_FILE=$HOME/.docker/config.json
```
    The authentication file is typically located in $HOME/.docker/config.json on Linux or %USERPROFILE%/.docker/config.json on Windows.
  2. Log in to the cp.icr.io registry with Podman or Docker; for example:
```
podman login cp.icr.io
```
    Use cp as the username and your entitlement key as the password.
Authenticate with the local registry.
Log in to the local registry using an account that can write images to that registry; for example:
```
podman login $TARGET_REGISTRY
```
If the registry is insecure, add the following flag to the command: --tls-verify=false.

Mirror the product images.

Connect the bastion host to both the internet and the restricted-network environment that contains the local registry.

Run the following commands to copy the images to the local registry:

API Connect:

oc image mirror \
  -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/images-mapping.txt \
  -a $REGISTRY_AUTH_FILE \
  --filter-by-os '.*' \
  --skip-multiple-scopes \
  --max-per-registry=1

Foundational services:

oc image mirror \
  -f ~/.ibm-pak/data/mirror/$CS_CASE_NAME/$CS_CASE_VERSION/images-mapping.txt \
  -a $REGISTRY_AUTH_FILE \
  --filter-by-os '.*' \
  --skip-multiple-scopes \
  --max-per-registry=1

EDB:

oc image mirror \
  -f ~/.ibm-pak/data/mirror/$EDB_CASE_NAME/$EDB_CASE_VERSION/images-mapping.txt \
  -a $REGISTRY_AUTH_FILE \
  --filter-by-os '.*' \
  --skip-multiple-scopes \
  --max-per-registry=1

Note: If the local registry does not use TLS, or your device does not trust the certificate that the local registry presents, add the --insecure option to the command.

There might be a slight delay before you see a response to the command.

Configure the target cluster.

Now that images are mirrored to the local registry, the target cluster must be configured to pull the images from it. Complete the following steps to configure the cluster's global pull secret with the local registry's credentials and then instruct the cluster to pull the images from the local registry.
1. Log in to your Red Hat OpenShift Container Platform cluster:
```
oc login <openshift_url> -u <username> -p <password> -n <namespace>
```
2. Update the global image pull secret for the cluster as explained in the Red Hat OpenShift Container Platform documentation.
  Updating the image pull secret provides the cluster with the credentials that are needed for pulling images from your local registry.
  Note: If you have an insecure registry, add the registry to the cluster's insecureRegistries list by running the following command:
```
oc edit image.config.openshift.io/cluster -o yaml
```
  and add the TARGET_REGISTRY to spec.registrySources.insecureRegistries as shown in the following example:
```
spec:
  registrySources:
    insecureRegistries:
    - insecure0.svc:5001
    - <TARGET_REGISTRY>
```
  If the insecureRegistries field does not exist, you can add it.
3. Create the ImageContentSourcePolicy, which instructs the cluster to pull the images from your local registry:
  - API Connect:
```
oc apply -f ~/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/image-content-source-policy.yaml
```
  - Foundational services:
```
oc apply -f ~/.ibm-pak/data/mirror/$CS_CASE_NAME/$CS_CASE_VERSION/image-content-source-policy.yaml
```
  - EDB:
```
oc apply -f ~/.ibm-pak/data/mirror/$EDB_CASE_NAME/$EDB_CASE_VERSION/image-content-source-policy.yaml
```
4. Verify that each ImageContentSourcePolicy resource was created:
```
oc get imageContentSourcePolicy
```
5. Verify your cluster node status:
```
oc get MachineConfigPool -w
```
  Wait for all nodes to be updated before you proceed to the next step.
Apply the catalog sources.

Now that the images are mirrored to the target cluster, apply the catalog sources for API Connect and Foundational services. You do not need to apply the catalog source for EDB because the API Connect operator applies that catalog source in its own namespace when the API Connect operand is created.

In the following steps, replace <Architecture> with either amd64 or s390x according to your environment.
1. Export the variables for the command line to use:
```
export CASE_NAME=ibm-apiconnect
export CASE_VERSION=5.3.0
export ARCH=amd64
```
```
export CS_CASE_NAME=ibm-cp-common-services
export CS_CASE_VERSION=4.6.3
export CS_ARCH=amd64
```
2. Generate the catalog source and save it in another directory in case you need to replicate this installation in the future.
  Get the catalog source:
```
cat ~/.ibm-pak/data/mirror/${CASE_NAME}/${CASE_VERSION}/catalog-sources.yaml
```
```
cat ~/.ibm-pak/data/mirror/${CS_CASE_NAME}/${CS_CASE_VERSION}/catalog-sources.yaml
```
  You can also navigate to the directory in your file browser to copy these artifacts into files that you can keep for re-use or for pipelines.
3. Apply the catalog sources to the cluster.
  1. Apply the universal catalog sources:
```
oc apply -f ~/.ibm-pak/data/mirror/${CASE_NAME}/${CASE_VERSION}/catalog-sources.yaml
```
```
oc apply -f ~/.ibm-pak/data/mirror/${CS_CASE_NAME}/${CS_CASE_VERSION}/catalog-sources.yaml
```
  2. Confirm that the catalog source was created in the openshift-marketplace namespace:
```
oc get catalogsource -n openshift-marketplace
```

What to do next

Upgrade your API Connect deployment, following the procedure documented in Upgrading API Connect on OpenShift.