Installing User Data Services in an air-gapped environment

You must install IBM Cloud Pak foundational services before you install the IBM User Data Services.

For more information about installing foundational services in an air-gapped environment, see Installing IBM Cloud Pak foundational services in an air-gapped environment.

After installing the foundational services, create an OperandRequest to install Events Operator as UDS has a dependency on it. See dependency here.

Note : IBM UDS operator requires certified-operators catalog source in READY state (for both online and offline installation) as it installs RedHat Certified Crunchy Postgres for Kubernetes as a dependency.

If you do not have the index image mirrored for certified-operators, you can prune the index image to your target registry and use it to create certified-operators catalog source.

opm index prune -f registry.redhat.io/redhat/certified-operator-index:v4.6 -p crunchy-postgres-operator -t <target-registry>/certified-operators:v4.6

Then, complete these steps to install the IBM User Data Services.

• 1. Set environment variables and download CASE files
• 2. Mirror images to portable registry
  • 2.1. Mirror the images to the host
  • 2.2. Copy saved offline data (for a portable storage device only)
  • 2.3. Connect your host to your air-gapped environment and set up your container
  • 2.4. Mirror images to final location and configure the cluster
• 3. Install the IBM User Data Services Operator

1. Set environment variables and download CASE files

Set the environment variables on your mirroring device, and connect to the internet so that you can download the corresponding CASE files. To finish preparing your host, complete the following steps:

1. Create the following environment variables with the IBM User Data Services operator image names and the image inventory on your host.

   On a bastion host:

    export CASE_NAME_UDS=ibm-uds
    export CASE_VERSION_UDS=2.0.1
    export CASE_ARCHIVE_UDS=$CASE_NAME_UDS-$CASE_VERSION_UDS.tgz
    export CASE_INVENTORY_SETUP_UDS=operatorSetup
    export OFFLINEDIR=$HOME/offline
    export CASE_REPO_PATH=https://github.com/IBM/cloud-pak/raw/master/repo/case
    export CASE_LOCAL_PATH_UDS=$OFFLINEDIR/$CASE_ARCHIVE_UDS
   

   On a portable compute device or portable storage device:

    export CASE_NAME_UDS=ibm-uds
    export CASE_VERSION_UDS=2.0.1
    export CASE_ARCHIVE_UDS=$CASE_NAME_UDS-$CASE_VERSION_UDS.tgz
    export CASE_INVENTORY_SETUP_UDS=operatorSetup
    export OFFLINEDIR=$HOME/offline
    export OFFLINEDIR_ARCHIVE_UDS=offline_uds.tgz
    export CASE_REPO_PATH=https://github.com/IBM/cloud-pak/raw/master/repo/case
    export CASE_LOCAL_PATH_UDS=$OFFLINEDIR/$CASE_ARCHIVE_UDS
   
    export PORTABLE_DOCKER_REGISTRY_HOST=localhost
    export PORTABLE_DOCKER_REGISTRY_PORT=443
    export PORTABLE_DOCKER_REGISTRY=$PORTABLE_DOCKER_REGISTRY_HOST:$PORTABLE_DOCKER_REGISTRY_PORT
    export PORTABLE_DOCKER_REGISTRY_USER=localuser
    export PORTABLE_DOCKER_REGISTRY_PASSWORD=l0calPassword!
    export PORTABLE_DOCKER_REGISTRY_PATH=$OFFLINEDIR/imageregistry
   

2. Connect your host to the internet and disconnect it from the local air-gapped network.

3. Download the IBM User Data Services operator image and image inventory to your host.

   oc ibm-pak case save \
     --repo $CASE_REPO_PATH \
     --case $CASE_NAME_UDS \
     --version $CASE_VERSION_UDS \
     --outputdir $OFFLINEDIR
   

Your host is now configured and you are ready to mirror your images.

Important: For portable compute and portable storage devices only, a Docker registry service must run from your connected device (localhost) by completing the following steps:

1. Initialize the Docker registry:

    oc ibm-pak case launch \
      --case $CASE_LOCAL_PATH_UDS \
      --inventory $CASE_INVENTORY_SETUP_UDS \
      --action init-registry \
      --args "--registry $PORTABLE_DOCKER_REGISTRY_HOST --user $PORTABLE_DOCKER_REGISTRY_USER --pass $PORTABLE_DOCKER_REGISTRY_PASSWORD --dir $PORTABLE_DOCKER_REGISTRY_PATH"
   

2. Start the Docker registry:

    oc ibm-pak case launch \
      --case $CASE_LOCAL_PATH_UDS \
      --inventory $CASE_INVENTORY_SETUP_UDS \
      --action start-registry \
      --args "--registry $PORTABLE_DOCKER_REGISTRY_HOST --port $PORTABLE_DOCKER_REGISTRY_PORT --user $PORTABLE_DOCKER_REGISTRY_USER --pass $PORTABLE_DOCKER_REGISTRY_PASSWORD --dir $PORTABLE_DOCKER_REGISTRY_PATH"
   

2. Mirror images to portable registry

The process of mirroring images takes the image from the internet to your host, then effectively copies that image on to your air-gapped environment. After you mirror your images, you can configure your cluster and complete air-gapped installation.

Complete the following steps to mirror your images from your host to your air-gapped environment:

• 2.1. Mirror the images to the host
• 2.2. Copy saved offline data (for a portable storage device only)
• 2.3. Connect your host to your air-gapped environment and set up your container
• 2.4. Mirror images to final location and configure the cluster

2.1. Mirror the images to the host

Complete these steps to mirror the images from the internet to your host:

Note: Don't use the tilde within double quotation marks in any command. For example, don't use args "--registry <registry> --user <registry userid> --pass {registry password} --inputDir ~/offline". The tilde does not expand and your commands might fail.

1. Store authentication credentials for all source Docker registries.

   All IBM Cloud Pak foundational services credentials are stored in public registries that don't require authentication. However, other products and third-party components require one or more authenticated registries. The following registries require authentication:

   • cp.icr.io
   • registry.redhat.io

   • registry.access.redhat.com

     For more information about these registries, see 1.2 Set up local image registry and access in the following topics:

   • Installing IBM Cloud Pak foundational services in an air-gapped environment using a Bastion host

   • Installing IBM Cloud Pak foundational services in an air-gapped environment using a portable compute device

   • Installing IBM Cloud Pak foundational services in an air-gapped environment using a portable storage device

     During foundational services installation, you must have the credentials for all registries that require authentication configured. For more information, see 3.1. Mirror the images to the host in the following topics:

   • Installing IBM Cloud Pak foundational services in an air-gapped environment using a Bastion host

   • Installing IBM Cloud Pak foundational services in an air-gapped environment using a portable compute device
   • Installing IBM Cloud Pak foundational services in an air-gapped environment using a portable storage device

2. Store authentication credentials of the host Docker registry:

   On a portable storage device:

    oc ibm-pak case launch \
      --case $CASE_LOCAL_PATH_UDS \
      --inventory $CASE_INVENTORY_SETUP_UDS \
      --action configure-creds-airgap \
      --args "--registry $PORTABLE_DOCKER_REGISTRY --user $PORTABLE_DOCKER_REGISTRY_USER --pass $PORTABLE_DOCKER_REGISTRY_PASSWORD"
   

   The command stores and caches the registry credentials in a file on your file system in the $HOME/.airgap/secrets location.

3. Mirror the images to the registry on the host.

    oc ibm-pak case launch \
      --case $CASE_LOCAL_PATH_UDS \
      --inventory $CASE_INVENTORY_SETUP_UDS \
      --action mirror-images \
      --args "--registry $PORTABLE_DOCKER_REGISTRY --inputDir $OFFLINEDIR"
   

4. Save the Docker registry image.

If your air-gapped network doesn’t have a Docker registry image, you can save the image and copy it later to the host in your air-gapped environment:

docker save docker.io/library/registry:2.6 -o $PORTABLE_DOCKER_REGISTRY_PATH/registry-image.tar

2.2. Copy saved offline data (for a portable storage device only)

1. Connect the portable storage device, such as a USB drive or external HDD, to this external host.

2. Archive the offline data for transfer:

    tar -cvzf $OFFLINEDIR_ARCHIVE_UDS -C $OFFLINEDIR .
   

3. Copy the preceding TAR file to the portable storage.

   Physically transfer your portable storage device from the machine that has a public internet connection to the machine that has no internet connectivity (your air-gapped environment).

4. Proceed to the next section to set up your cluster.

2.3. Connect your host to your air-gapped environment and set up your container

1. Connect your host device to the air-gapped network and disconnect it from the internet.

2. Log in to the OpenShift Container Platform cluster as a cluster administrator. The following sample command logs in to the OpenShift Container Platform cluster:

    oc login <cluster host:port> --username=<cluster admin user> --password=<cluster admin password>
   

You must install the IBM User Data Services Operator in the same namespace where you installed foundational services. The NAMESPACE environment variable already has the this namespace value from your foundational services installation.

2.4. Mirror images to final location and configure the cluster

Complete these steps on your host that is connected to both the local Docker registry and the OpenShift Container Platform cluster:

1. Create environment variables with the local Docker registry connection information.

   On a bastion host or portable compute device:

    export CASE_NAME_UDS=ibm-uds
    export CASE_VERSION_UDS=2.0.1
    export CASE_ARCHIVE_UDS=$CASE_NAME_UDS-$CASE_VERSION_UDS.tgz
    export CASE_INVENTORY_SETUP_UDS=operatorSetup
    export OFFLINEDIR=$HOME/offline
    export OFFLINEDIR_ARCHIVE_UDS=offline_uds.tgz
    export CASE_REPO_PATH=https://github.com/IBM/cloud-pak/raw/master/repo/case
    export CASE_LOCAL_PATH_UDS=$OFFLINEDIR/$CASE_ARCHIVE_UDS
   
    export LOCAL_DOCKER_REGISTRY_HOST=<IP_or_FQDN_of_local_docker_registry>
    export LOCAL_DOCKER_REGISTRY_PORT=443
    export LOCAL_DOCKER_REGISTRY=$LOCAL_DOCKER_REGISTRY_HOST:$LOCAL_DOCKER_REGISTRY_PORT
    export LOCAL_DOCKER_USER=<username>
    export LOCAL_DOCKER_PASSWORD=<password>
   

   On a portable storage device:

    export CASE_NAME_UDS=ibm-uds
    export CASE_VERSION_UDS=2.0.1
    export CASE_ARCHIVE_UDS=$CASE_NAME_UDS-$CASE_VERSION_UDS.tgz
    export CASE_INVENTORY_SETUP_UDS=operatorSetup
    export OFFLINEDIR=$HOME/offline
    export OFFLINEDIR_ARCHIVE_UDS=offline_uds.tgz
    export CASE_REPO_PATH=https://github.com/IBM/cloud-pak/raw/master/repo/case
    export CASE_LOCAL_PATH_UDS=$OFFLINEDIR/$CASE_ARCHIVE_UDS
   
    export PORTABLE_DOCKER_REGISTRY_HOST=localhost
    export PORTABLE_DOCKER_REGISTRY_PORT=443
    export PORTABLE_DOCKER_REGISTRY=$PORTABLE_DOCKER_REGISTRY_HOST:$PORTABLE_DOCKER_REGISTRY_PORT
    export PORTABLE_DOCKER_REGISTRY_USER=localuser
    export PORTABLE_DOCKER_REGISTRY_PASSWORD=l0calPassword!
    export PORTABLE_DOCKER_REGISTRY_PATH=$OFFLINEDIR/imageregistry
   
    export LOCAL_DOCKER_REGISTRY_HOST=<IP_or_FQDN_of_local_docker_registry>
    export LOCAL_DOCKER_REGISTRY_PORT=443
    export LOCAL_DOCKER_REGISTRY=$LOCAL_DOCKER_REGISTRY_HOST:$LOCAL_DOCKER_REGISTRY_PORT
    export LOCAL_DOCKER_REGISTRY_USER=<username>
    export LOCAL_DOCKER_REGISTRY_PASSWORD=<password>
   

   Note: The Docker registry uses standard ports such as 80 or 443. If your Docker registry uses a non-standard port, specify the port by using the syntax <host>:<port>. For example, export LOCAL_DOCKER_REGISTRY=myregistry.local:5000.

2. On a portable storage device only, extract the transferred offline data:

    mkdir -p $OFFLINEDIR
    tar -xvf $OFFLINEDIR_ARCHIVE_UDS -C $OFFLINEDIR
   

3. On a portable storage device only, set up the registry. Run the local Docker registry as a container. The registry then points to the Docker file system directory that is transferred from the external host:

    oc ibm-pak case launch \
      --case $CASE_LOCAL_PATH_UDS \
      --inventory $CASE_INVENTORY_SETUP_UDS \
      --action start-registry \
      --args "--registry $PORTABLE_DOCKER_REGISTRY_HOST --port $PORTABLE_DOCKER_REGISTRY_PORT --user $PORTABLE_DOCKER_REGISTRY_USER --pass $PORTABLE_DOCKER_REGISTRY_PASSWORD --dir $PORTABLE_DOCKER_REGISTRY_PATH"
   

4. Configure an authentication secret for the Docker registry.

Note: This step needs to be done only one time.

On a bastion host or portable compute device:

   oc ibm-pak case launch \
     --case $HOME/offline/$CASE_ARCHIVE_UDS \
     --inventory $CASE_INVENTORY_SETUP_UDS \
     --action configure-creds-airgap \
     --args "--registry $LOCAL_DOCKER_REGISTRY --user $LOCAL_DOCKER_USER --pass $LOCAL_DOCKER_PASSWORD"

On a portable storage device, store credentials of the registry that is running on the internal host (created in the previous step):

   oc ibm-pak case launch \
     --case $CASE_LOCAL_PATH_UDS \
     --inventory $CASE_INVENTORY_SETUP_UDS \
     --action configure-creds-airgap \
     --args "--registry $PORTABLE_DOCKER_REGISTRY --user $PORTABLE_DOCKER_REGISTRY_USER --pass $PORTABLE_DOCKER_REGISTRY_PASSWORD"

Then, store credentials of the registry that is going to serve images to the cluster/workloads:

   oc ibm-pak case launch \
     --case $CASE_LOCAL_PATH_UDS \
     --inventory $CASE_INVENTORY_SETUP_UDS \
     --action configure-creds-airgap \
     --args "--registry $LOCAL_DOCKER_REGISTRY --user $LOCAL_DOCKER_REGISTRY_USER --pass $LOCAL_DOCKER_REGISTRY_PASSWORD"

The command stores and caches the registry credentials in a file on your file system in the $HOME/.airgap/secrets location.

1. Mirror images to the local image registry:

   On a portable compute device or portable storage device, mirror the images from the portable registry to the target registry on your cluster.

    oc ibm-pak case launch \
      --case $CASE_LOCAL_PATH_UDS \
      --inventory $CASE_INVENTORY_SETUP_UDS \
      --action mirror-images \
      --args "--fromRegistry $PORTABLE_DOCKER_REGISTRY --registry $LOCAL_DOCKER_REGISTRY --inputDir $OFFLINEDIR"
   

   On a bastion server:

    oc ibm-pak case launch \
      --case $CASE_LOCAL_PATH_UDS \
      --inventory $CASE_INVENTORY_SETUP_UDS \
      --action mirror-images \
      --args "--registry $LOCAL_DOCKER_REGISTRY --inputDir $OFFLINEDIR"
   

2. Configure a global image pull secret and ImageContentSourcePolicy.

    oc ibm-pak case launch \
      --case $HOME/offline/$CASE_ARCHIVE_UDS \
      --inventory $CASE_INVENTORY_SETUP_UDS \
      --action configure-cluster-airgap \
      --namespace $NAMESPACE \
      --args "--registry $LOCAL_DOCKER_REGISTRY --user $LOCAL_DOCKER_USER --pass $LOCAL_DOCKER_PASSWORD --inputDir $OFFLINEDIR"
   

3. Verify that the ImageContentSourcePolicy resource is created.

    oc get imageContentSourcePolicy
   

4. Optional: If you use an insecure registry, you must add the local registry to the cluster insecureRegistries list.

    oc patch image.config.openshift.io/cluster --type=merge \
    -p '{"spec":{"registrySources":{"insecureRegistries":["'${LOCAL_DOCKER_REGISTRY}'"]}}}'
   

5. Verify your cluster node status.

    oc get nodes
   

After the imageContentsourcePolicy and global image pull secret are applied, you might see the node status as Ready, Scheduling, or Disabled. Wait until all the nodes show a Ready status.

3. Install the IBM User Data Services Operator

Now that your images are mirrored to your air-gapped environment, you can deploy the IBM User Data Services Operator to that environment. Complete the following steps:

1. Create and configure a catalog source.

    oc ibm-pak case launch \
      --case $CASE_LOCAL_PATH_UDS \
      --inventory $CASE_INVENTORY_SETUP_UDS \
      --action install-catalog \
      --namespace $NAMESPACE \
      --args "--registry $LOCAL_DOCKER_REGISTRY"
   

2. Verify that the CatalogSource is created.

    oc get pods -n openshift-marketplace
    oc get catalogsource -n openshift-marketplace
    oc get catalogsource ibm-udsoperator-catalog -n openshift-marketplace -o yaml
   

3. Install the IBM User Data Services Operator.

    oc ibm-pak case launch \
      --case $CASE_LOCAL_PATH_UDS \
      --inventory $CASE_INVENTORY_SETUP_UDS \
      --action install-operator \
      --namespace $NAMESPACE
   

4. Verify that the service is installed:

    oc get pod -n <your-foundational-services-namespace>
   

5. Change the installPlanApproval of IBM User Data Services to Manual by patching the subscription.

    oc patch subscription ibm-user-data-services-operator \
      --namespace $NAMESPACE \
      --type merge \
      --patch '{"spec":{"installPlanApproval":"Manual"}}'
   

6. Create an instance of AnalyticsProxy or AnalyticsProxyWithSubmodule custom resource (CR).

   • To check the AnalyticsProxy or AnalyticsProxyWithSubmodule CR attribute details, see the list of configurable parameters.

   • To check more details on air-gapped installation, see offline installation document.

    oc ibm-pak case launch \
      --case $CASE_DIR   \
      --namespace $NS    \
      --inventory operator  \
      --action apply-analyticsproxy \
      --tolerance 1  \
      --args "--accept_license true \
        --airgap_enabled true \
        --event_scheduler_frequency @hourly \
        --ibm_proxy_url https://iaps.ibm.com \
        --postgres_storage_size 10G \
        --postgres_backup_type incremental \
        --postgres_backup_frequency @daily \
        --storage_class ibmc-block-bronze \
        --kafka_storage_size 5G \
        --kafka_Zookeeper_storage_size 5G \
        --env_type lite \
        --dba_storage_size 10G \
        --db_type internal "
   

   OR

    oc ibm-pak case launch \
      --case $CASE_DIR   \
      --namespace $NS    \
      --inventory operator  \
      --action apply-analyticsproxy-wsm \
      --tolerance 1  \
      --args "--accept_license true \
        --airgap_enabled true \
        --submodule_scheduler_frequency @daily \
        --image_pull_secret uds_image_pull_secret
        --event_scheduler_frequency @hourly \
        --ibm_proxy_url https://dev.iaps.ibm.com \
        --postgres_storage_size 10G \
        --postgres_backup_type incremental \
        --postgres_backup_frequency @daily \
        --storage_class ibmc-block-bronze \
        --kafka_storage_size 5G \
        --kafka_Zookeeper_storage_size 5G \
        --env_type lite \
        --dba_storage_size 10G \
        --db_type internal "
   

7. After you create Analytics Proxy instance, wait for Crunchy Postgres operator to install. Once the subscription for Crunchy Postgres operator is created, patch it to change the installPlanApproval to Manual.

    oc patch subscription crunchy-postgres-operator \
     --namespace $NAMESPACE \
     --type merge \
     --patch '{"spec":{"installPlanApproval":"Manual"}}'
   

8. Approve the installplan of Crunchy Postgres operator to install the operator.

    oc get installplan
   

    oc patch installplan <postgresoperator-installplan-name> \
      --namespace $NAMESPACE \
      --type merge \
      --patch '{"spec":{"approved":true}}'
   

9. Create an instance of GenerateKey.

    oc ibm-pak case launch  \
      --case $CASE_LOCAL_PATH_UDS    \
      --namespace $NAMESPACE  \
      --inventory operator \
      --action generate-api-key \
      --args "--key_name uds-api-key
   

You now successfully created and deployed your air-gapped instance of IBM User Data Services Operator.