cp-console address is changed after CS operator is upgraded to v4, but IAM service is still in v3

The cp-console address is changed immediately after IBM Common Service operator is upgraded to v4 but IAM service is still in v3.

Symptom

The cp-console address is changed with a new namespace suffix immediately after the IBM Common Service operator is upgraded to v4, while IBM IAM service remains in v3.x level. The old cp-console URL is no longer accessible.

The cp-console address is without the namespace suffix before the Common Service upgrade.

> oc get route -n ibm-common-services
NAME         HOST/PORT                                          PATH   SERVICES                   PORT    TERMINATION            WILDCARD
cp-console   cp-console.apps.installer-cp3pt0.cp.fyre.ibm.com          icp-management-ingress     https   reencrypt/Redirect     None

The cp-console address is updated with namespace suffix after the Common Service Operator is upgraded while the IBM IAM service remains in v3.x.

> oc get csv | grep 'ibm-common-service-operator\|ibm-iam-operator'
ibm-common-service-operator.v4.3.0              IBM Cloud Pak foundational services    4.3.0   ibm-common-service-operator.v3.23.9           Succeeded
ibm-iam-operator.v3.23.10                       IBM IAM                                3.23.10   ibm-iam-operator.v3.23.9                       Succeeded
> oc get route -n ibm-common-services
NAME           HOST/PORT                                                             PATH   SERVICES                   PORT    TERMINATION            WILDCARD
cp-console    cp-console-ibm-common-services.apps.installer-cp3pt0.cp.fyre.ibm.com          icp-management-ingress     https   reencrypt/Redirect     None

Cause

The Common Service Operator v4 updates the OperandConfig with an improper configuration for IBM IAM service v3.x, causing the cp-console change.

The correct behavior is that the cp-console address should only be changed with the new namespace suffix only after IBM IAM service is upgraded to v4.x level.

> oc get csv | grep ibm-iam-operator
ibm-iam-operator.v4.3.0                       IBM IM Operator                        4.3.0                                       Succeeded
> oc get route -n ibm-common-services
NAME                    HOST/PORT                                                              PATH                        SERVICES                       PORT                   TERMINATION            WILDCARD
cp-console              cp-console-ibm-common-services.apps.installer-cp3pt0.cp.fyre.ibm.com   /                           common-web-ui                  3000                   reencrypt/Redirect     None

Workaround

You can manually patch CommonService CR and OperandConfig to revert the cp-console route back to URL without the namespace suffix. Here is a template in CommonService CR:

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
spec:
  services:
    - name: ibm-management-ingress-operator
      spec:
        managementIngress:
          multipleInstancesEnabled: false

1. Make sure that the jq binary is available in your local machine.

    jq --version
   

2. Update the CommonService CR to explicitly set the preceding template.

    operatorNamespace=<your-foundational-operator-namespace>
    oc get commonservice common-service -n $operatorNamespace -o json |
      jq '.spec.services |= (if type == "array" then map(if .name == "ibm-management-ingress-operator" then .spec.managementIngress.multipleInstancesEnabled = false else . end) else . end // []) |
        if (.spec.services | map(.name == "ibm-management-ingress-operator") | any == false) then .spec.services += [{"name": "ibm-management-ingress-operator", "spec": {"managementIngress": {"multipleInstancesEnabled": false}}}] else . end' |
      oc apply -f -
   

3. Find the services namespace for foundational services.

    servicesNamespace=$(oc get commonservice common-service -n $operatorNamespace -o jsonpath='{.spec.servicesNamespace}')
   

4. Update the OperandConfig in the services namespace to explicitly set the preceding template.

    oc get operandconfig common-service -n $servicesNamespace -o json |
      jq '.spec.services |= (if type == "array" then map(if .name == "ibm-management-ingress-operator" then .spec.managementIngress.multipleInstancesEnabled = false else . end) else . end // []) |
        if (.spec.services | map(.name == "ibm-management-ingress-operator") | any == false) then .spec.services += [{"name": "ibm-management-ingress-operator", "spec": {"managementIngress": {"multipleInstancesEnabled": false}}}] else . end' |
      oc apply -f -
   

5. Wait for the cp-console route to be updated without a namespace suffix.

    oc get route -n $servicesNamespace
    NAME         HOST/PORT                                          PATH   SERVICES                   PORT    TERMINATION            WILDCARD
    cp-console   cp-console.apps.installer-cp3pt0.cp.fyre.ibm.com          icp-management-ingress     https   reencrypt/Redirect     None