• Business Automation Insights 24.0.0 IF004
• Cloud Pak Foundational Services 4.6.16

git clone -b 24.0.0-IF004 https://github.com/icp4a/cert-kubernetes-bai.git

• Scenario 1: You are installing a Production deployment
  Actions: 
  You can use this interim fix content to perform a Production deployment.  To deploy a Production deployment using the content of this interim fix, please see Installing a Business Automation Insights production environment  and use the CASE package from this interim fix.
  Note: If you have an existing Cloud Pak Foundation Services instance installed in the cluster or in the namespace where BAI is being installed, then it is not supported.  The Production deployment of BAI is only supported when deploying into a new namespace without CPFS.

• Scenario 2:  Your installed Production deployment is 24.0.0 GA and is online.
  Warning:
  For a Business Automation Insights deployment, it is recommended to create BAI savepoints before starting the upgrade process to this interim fix. For Flink event processing to resume from its previous state, savepoints are required to be created before the upgrade and specified in the updated CR. BAI savepoints can be created by following the below steps.
  • Retrieve the name of the InsightsEngines custom resource file

    INSIGHTS_ENGINE_CR=$(kubectl get insightsengines.bai.ibm.com --no-headers --ignore-not-found -n <BAI-Namespace> -o name)

  • Retrieve and export the below details.

    export MANAGEMENT_URL=$(kubectl get ${INSIGHTS_ENGINE_CR} --no-headers --ignore-not-found -n <BAI Namespace> -o jsonpath='{.status.components.management.endpoints[?(@.scope=="External")].uri}')
    export MANAGEMENT_AUTH_SECRET=$(kubectl get ${INSIGHTS_ENGINE_CR} --no-headers --ignore-not-found -n <BAI Namespace> -o jsonpath='{.status.components.management.endpoints[?(@.scope=="External")].authentication.secret.secretName}')
    export MANAGEMENT_USERNAME=$(kubectl get secret ${MANAGEMENT_AUTH_SECRET} --no-headers --ignore-not-found -n <BAI Namespace> -o jsonpath='{.data.username}' | base64 -d)
    export MANAGEMENT_PASSWORD=$(kubectl get secret ${MANAGEMENT_AUTH_SECRET} --no-headers --ignore-not-found -n <BAI Namespace> -o jsonpath='{.data.password}' | base64 -d)

  • Create BAI savepoints and store them in a temporary file called bai.json.

    curl -X POST -k -u ${MANAGEMENT_USERNAME}:${MANAGEMENT_PASSWORD} "${MANAGEMENT_URL}/api/v1/processing/jobs/savepoints" -o ./bai.json

  • Scale down the Insights Engine Operator.

    oc scale --replicas=0 deployment ibm-bai-insights-engine-operator

  • Retrieve the recovery path locations for each BAI component for which BAI savepoints are created from ./bai.json and update the bai_configuration section of the custom resource file. For Example: If there is a BAI savepoint being created for navigator component, then the updated custom resource file should have the below configuration.

    bai_configuration:
          navigator:
            recovery_path: /mnt/pv/savepoints/dba/bai-navigator/savepoint-fb88f4-42027046b73b
          ... 
          # Add recovery_path for all other components

  • Once the upgrade has been completed, make sure to remove all instances of the recovery_path parameters from the updated custom resource file.
  Actions: 
  Perform the following steps and then the upgrade of operators and deployments will start.
  1. Upgrade the IBM Business Automation Insights operators using one of two methods. 
     • Option 1: Running the operator upgrade script from the case package.

       ./scripts/bai-deployment.sh -m upgradeOperator -n <project_name>

     • Option 2: Manually deploy the catalog source and update the CPFS channels using the CPFS upgrade script.
       1. Apply the new catalog sources.

          oc apply -f ./descriptors/op-olm/catalog_source.yaml

       2. Update the cert manager and license service channels to the appropriate level with this script:

          ./scripts/cpfs/installer_scripts/cp3pt0-deployment/setup_singleton.sh --enable-licensing --cert-manager-source ibm-cert-manager-catalog --licensing-source ibm-licensing-catalog --license-accept -v 1 -c v4.2

       3. Update the channels to the appropriate levels for the rest of the CPFS subscriptions.

          ./scripts/cpfs/installer_scripts/cp3pt0-deployment/setup_tenant.sh --operator-namespace <BAI Namespace> -s ibm-cs-install-catalog-v4-6-9 -c v4.6 --license-accept -v 1

          Note: Be sure to replace the namespace with the appropriate BAI namespace.
  2. Wait for the operators to complete their upgrades.
     By default all subscriptions are set to automatic, but if you have any subscriptions set to manual then you need to approve any pending InstallPlans.
     Use the below command to see the current status of the install plans.

     oc get installPlan

     The upgrade will be blocked, if any of the needed InstallPlans are not approved. It is not recommended to set subscriptions to manual as this makes the upgrade more error prone.
  3. Check the status of the upgrades by running the following scripts:
     1. Run the script in [upgradeOperatorStatus] mode to check that the upgrade of the BAI operator and its dependencies is successful.

        ./scripts/bai-deployment.sh -m upgradeOperatorStatus -n <project_name>

        Warning: The script will scale the BAI deployments down to zero. You must execute the upgradeDeploymentStatus command to scale them back up.
     2. Run the script in [upgradeDeploymentStatus] mode to check that the upgrade of the BAI deployment is successful.

        ./scripts/bai-deployment.sh -m upgradeDeploymentStatus -n <project_name>

• Scenario 3: You are installing offline/airgap Production deployment
  Warning: If you have an existing Cloud Pak Foundation Services instance installed at the cluster scoped level or in the namespace where BAI is being installed, then it is not supported. The Production deployment of BAI is only supported when deploying into a new namespace without CPFs.
  Note: As prerequisites for this scenario, you must follow steps here to set up the bastion host to mirror images to the registry and further to set up the private registry. The mirroring of images can be completed using "oc mirror" for the mirroring images process. 
  Actions:
  1. To deploy an airgap/offline Production deployment, find mirror file bai-case-to-be-mirrored-24.0.0-IF004.txt for this interim fix from the branch that you cloned above under the scripts/airgap directory. Execute this command from your bastion host to download the CASE files.
     Execute this command from your bastion host to download the CASE files.

     oc ibm-pak get -c file://(absolute path to file)/bai-case-to-be-mirrored-24.0.0-IF004.txt

     The absolute path to file needs to be a path starting from "/". For example, "/opt".
  2. Mirror the images associated with the new bai-case-to-be-mirrored-24.0.0-IF004.txt mirror file. 

     export CASE_NAME=ibm-ba-insights
     export CASE_VERSION=24.0.4
     export CASE_INVENTORY_SETUP=baiOperatorSetup
     export TARGET_REGISTRY=<target-registry>
     export NAMESPACE=<bai_namespace_name>

     Follow the instructions for either mirroring option in Mirroring images to the private registry using the new CASE version associated with this interim fix.
  3. Login to the cluster and go to namespace for the operator from the bastion host.

     oc login https://<CLUSTERIP>:<port> -u <ADMINISTRATOR>
     oc project ${NAMESPACE}
     

  4. From your bastion host, install the catalog sources and operators using the steps listed in Installing the Business Automation Insights catalog and operator instances in an off-line cluster.

• Scenario 4:  Your installed Production deployment is 24.0.0 GA and using airgap/offline.
  Note: As prerequisites for this scenario, you must follow steps here to set up the bastion host to mirror images to the registry and further to set up the private registry. 
  Warning:
  For a Business Automation Insights deployment, it is recommended to create BAI savepoints before starting the upgrade process to this interim fix. For Flink event processing to resume from its previous state, savepoints are required to be created before the upgrade and specified in the updated CR. BAI savepoints can be created by following the below steps.
  •  Retrieve the name of the InsightsEngines custom resource file

    INSIGHTS_ENGINE_CR=$(kubectl get insightsengines.bai.ibm.com --no-headers --ignore-not-found -n <BAI-Namespace> -o name)

  • Retrieve and export the below details.

    export MANAGEMENT_URL=$(kubectl get ${INSIGHTS_ENGINE_CR} --no-headers --ignore-not-found -n <BAI Namespace> -o jsonpath='{.status.components.management.endpoints[?(@.scope=="External")].uri}')
    export MANAGEMENT_AUTH_SECRET=$(kubectl get ${INSIGHTS_ENGINE_CR} --no-headers --ignore-not-found -n <BAI Namespace> -o jsonpath='{.status.components.management.endpoints[?(@.scope=="External")].authentication.secret.secretName}')
    export MANAGEMENT_USERNAME=$(kubectl get secret ${MANAGEMENT_AUTH_SECRET} --no-headers --ignore-not-found -n <BAI Namespace> -o jsonpath='{.data.username}' | base64 -d)
    export MANAGEMENT_PASSWORD=$(kubectl get secret ${MANAGEMENT_AUTH_SECRET} --no-headers --ignore-not-found -n <BAI Namespace> -o jsonpath='{.data.password}' | base64 -d)

  • Create BAI savepoints and store them in a temporary file called bai.json.

    curl -X POST -k -u ${MANAGEMENT_USERNAME}:${MANAGEMENT_PASSWORD} "${MANAGEMENT_URL}/api/v1/processing/jobs/savepoints" -o ./bai.json

  • Scale down the Insights Engine Operator.

    oc scale --replicas=0 deployment ibm-bai-insights-engine-operator

  • Retrieve the recovery path locations for each BAI component for which BAI savepoints are created from ./bai.json and update the bai_configuration section of the custom resource file. For Example: If there is a BAI savepoint being created for navigator component, then the updated custom resource file should have the below configuration.

  • bai_configuration:
          navigator:
            recovery_path: /mnt/pv/savepoints/dba/bai-navigator/savepoint-fb88f4-42027046b73b
          ... 
          # Add recovery_path for all other components

  • Once the upgrade has been completed, make sure to remove all instances of the recovery_path parameters from the updated custom resource file.
  Actions: 
  Perform the following steps and then the upgrade of operators and deployments will start. 
  1.  Find mirror file  bai-case-to-be-mirrored-24.0.0-IF004.txt for this interim fix from the branch that you cloned above under the scripts/airgap directory. Execute this command from your bastion host to download the CASE files:

     oc ibm-pak get -c file://(absolute path to file)/bai-case-to-be-mirrored-24.0.0-IF004.yaml

     The (absolute path to file) needs to be a path starting from "/". For example, "/opt"
  2. Mirror the images associated with the new bai-case-to-be-mirrored-24.0.0-IF004.txt mirror file. 

     export CASE_NAME=ibm-ba-insights
     export CASE_VERSION=24.0.4
     export CASE_INVENTORY_SETUP=baiOperatorSetup
     export TARGET_REGISTRY=<target-registry>
     export NAMESPACE=<bai_namespace_name>

     Follow the instructions for either mirroring option in Mirroring images to the private registry using the new CASE version associated with this interim fix.
  3.  From the branch that you cloned above navigate to the scripts directory and perform the following steps to upgrade the BAI operators.

     Warning: 
     The script with the upgradeOperator option will scale the BAI Operators down to zero. You must execute the script with the upgradeDeploymentStatus mode to scale them back up. 

     Actions:
     Run the bai-deployment.sh script with the upgradeOperator option to upgrade the IBM Cloud Pak foundational services/BAI operators:

     ./scripts/bai-deployment.sh -m upgradeOperator -n <BAI Namespace>

  4. Wait for the operators to complete their upgrades.

     By default all subscriptions are set to automatic, but if you have any subscriptions set to manual then you need to approve any pending InstallPlans.
     Use the below command to see the current status of the install plans.

     oc get installPlan

     The upgrade will be blocked, if any of the needed InstallPlans are not approved. It is not recommended to set subscriptions to manual as this makes the upgrade more error prone.
  5. You can use the following scripts to check the status of the upgrades.

     • Warning:
       The script will scale the BAI deployments down to zero. You must execute the bai-deployment.sh script with upgradeDeploymentStatus option to scale them back up.
     • Actions:
       [OPTIONAL] Run the bai-deployment.sh script with upgradeOperatorStatus option to check that the upgrade of the BAI operator and its dependencies is successful:

     ./scripts/bai-deployment.sh -m upgradeOperatorStatus -n <BAI Namespace>

     Note: Make sure you replace the namespace with the appropriate BAI namespace.
  6. Start up the upgraded BAI Operators.
     Run the bai-deployment.sh script with upgradeDeploymentStatus option to check that the upgrade of the BAI deployment is successful:

     ./scripts/bai-deployment.sh -m upgradeDeploymentStatus -n <BAI Namespace>

Performing the necessary tasks after installation

Document change history