Upgrading IBM Software Hub (Upgrading from Version 5.1 to Version 5.3)

If you have any of the following services on this instance of IBM Software Hub, complete the required steps before you upgrade IBM Software Hub:

oc delete isvc ibm-granite-3b-code-v1 ibm-granite-20b-code-8k-ansible \
-n ${PROJECT_CPD_INST_OPERANDS} \
--ignore-not-found

1. Check whether the global search pods are running:

   oc get pods --namespace=${PROJECT_CPD_INST_OPERANDS} | grep elasticsea-0ac3

   • If the command returns an empty response, proceed to the next step.
   • If the command returns a list of pods, review Upgrades fail when global search is configured incorrectly to determine whether you have any configurations that could cause issues during upgrade.
2. Check whether the catalog-api pods are running:

   oc get pods --namespace=${PROJECT_CPD_INST_OPERANDS} | grep catalog-api

   • If the command returns an empty response, you are ready to upgrade IBM Software Hub.
   • If the command returns a list of pods, review the following guidance to determine how long the catalog-api service will be down during upgrade.

   When you upgrade the common core services to IBM Software Hub Version 5.3, the underlying storage for the catalog-api service is migrated to PostgreSQL.

   During the final stages of the migration, the catalog-api service is offline, and services that are dependent on the service are not available. The duration of the migration depends on the number of assets and relationships that are stored in the instance. The duration of the outage depends on the number of databases (projects, catalogs, and spaces) in the instance. In a typical upgrade scenario, the outage should be significantly shorter than the overall migration.

   To determine how many databases will be migrated:

   1. Set the INSTANCE_URL environment variable to the URL of IBM Software Hub:

      export INSTANCE_URL=<URL>

      Tip: To get the URL of the web client, run the following command:

      cpd-cli manage get-cpd-instance-details \
      --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS}

   2. Get the credentials for the wdp-service:

      TOKEN=$(oc get -n ${PROJECT_CPD_INST_OPERANDS} secrets wdp-service-id -o yaml | grep service-id-credentials | cut -d':' -f2- | sed -e 's/ //g' | base64 -d)

   3. Get the number of catalogs in the instance:

      curl -sk -X GET "https://${INSTANCE_URL}/v2/catalogs?limit=10001&skip=0&include=catalogs&bss_account_id=999" -H 'accept: application/json' -H "Authorization: Basic ${TOKEN}" | jq -r '.catalogs | length'

   4. Get the number of projects in the instance:

      curl -sk -X GET "https://${INSTANCE_URL}/v2/catalogs?limit=10001&skip=0&include=projects&bss_account_id=999" -H 'accept: application/json' -H "Authorization: Basic ${TOKEN}" | jq -r '.catalogs | length'

   5. Get the number of spaces in the instance:

      curl -sk -X GET "https://${INSTANCE_URL}/v2/catalogs?limit=10001&skip=0&include=spaces&bss_account_id=999" -H 'accept: application/json' -H "Authorization: Basic ${TOKEN}" | jq -r '.catalogs | length'

   6. Add up the number of catalogs, projects, and spaces returned by the previous commands. Then, use the following table to determine approximately how long the service will be offline during the migration:

      Databases	Downtime for migration (approximate)
      Up to 1,000 databases	6 minutes
      1,001 - 10,000 databases	20 minutes
      10,001 - 70,000 databases	60 minutes

3. Save the following script on the client workstation as a file named precheck_migration.sh:

   #!/bin/bash
   
   # Default ranges for couchdb size
   SMALL=50
   MEDIUM=100
   LARGE=200
   
   echo "Performing pre-migration checks"
   
   patch_for_small()
   {
     echo -e "Run the following command to increase the CPU and memory:\n"
             cat << EOF
   oc patch ccs ccs-cr -n ${PROJECT_CPD_INST_OPERANDS} --type merge --patch '{"spec": {
     "catalog_api_postgres_migration_threads": 4,
     "catalog_api_migration_job_resources": { 
       "requests": {"cpu": "2", "ephemeral-storage": "10Mi", "memory": "2Gi"},
       "limits": {"cpu": "6", "ephemeral-storage": "1Gi", "memory": "6Gi"}}
   }}'
   EOF
       echo
       echo "The system is ready for migration. Upgrade your cluster as usual."
   }
   
   patch_for_medium()
   {
       echo -e "Run the following command to increase the CPU and memory:\n"
             cat << EOF
   oc patch ccs ccs-cr -n ${PROJECT_CPD_INST_OPERANDS} --type merge --patch '{"spec": {
     "catalog_api_postgres_migration_threads": 6,
     "catalog_api_migration_job_resources": { 
       "requests": {"cpu": "3", "ephemeral-storage": "10Mi", "memory": "4Gi"},
       "limits": {"cpu": "8", "ephemeral-storage": "4Gi", "memory": "8Gi"}}
   }}'
   EOF
       echo
       echo "The system is ready for migration. Upgrade your cluster as usual."
   }
   
   patch_for_large()
   {
       echo -e "Run the following command to increase the CPU and memory:\n"
             cat << EOF
   oc patch ccs ccs-cr -n ${PROJECT_CPD_INST_OPERANDS} --type merge --patch '{"spec": {
     "catalog_api_postgres_migration_threads": 8,
     "catalog_api_migration_job_resources": { 
       "requests": {"cpu": "6", "ephemeral-storage": "10Mi", "memory": "6Gi"},
       "limits": {"cpu": "10", "ephemeral-storage": "6Gi", "memory": "10Gi"}}
   }}'
   EOF
       echo
       echo "Before you can start the upgrade, you must prepare the system for migration."
   }
   
   check_resources(){
           scale_config=$1
           #pvc_size=$(oc get pvc -n ${PROJECT_CPD_INST_OPERANDS} database-storage-wdp-couchdb-0 --no-headers | awk '{print $4}')
           #size=$(awk '{print substr($0, 1, length($0)-2)}' <<< "$pvc_size")
           size=20
   
           if [[ $scale_config == "small" ]];then
             if [[ "$size" -le "$SMALL" ]];then
               echo "The system is ready for migration. Upgrade your cluster as usual."
             elif [ "$size" -ge "$SMALL" ] && [ "$size" -le "$MEDIUM" ];then
               patch_for_medium
             elif [ "$size" -ge "$MEDIUM" ] && [ "$size" -le "$LARGE" ];then
               patch_for_large
             else
               patch_for_large
             fi
           elif [[ $scale_config == "medium" ]];then
             if [[ "$size" -le "$SMALL" ]];then
               patch_for_small
             elif [ "$size" -ge "$SMALL" ] && [ "$size" -le "$MEDIUM" ];then
               echo "The system is ready for migration. Upgrade your cluster as usual."
             elif [ "$size" -ge "$MEDIUM" ] && [ "$size" -le "$LARGE" ];then
               patch_for_large
             else
               patch_for_large
             fi
           elif [[ $scale_config == "large" ]];then
             if [[ "$size" -le "$SMALL" ]];then
               patch_for_small
             elif [ "$size" -ge "$SMALL" ] && [ "$size" -le "$MEDIUM" ];then
               patch_for_medium
             elif [ "$size" -ge "$MEDIUM" ] && [ "$size" -le "$LARGE" ];then
               echo "The system is ready for migration. Upgrade your cluster as usual."
             else
               patch_for_large
             fi
           fi
   }
   
   check_upgrade_case(){     
           echo -e "Checking if automatic upgrade or semi-automatic upgrade is needed"
           #scale_config=$(oc get ccs -n ${PROJECT_CPD_INST_OPERANDS} ccs-cr -o json | jq -r '.spec.scaleConfig')
           scale_config=large
   
           # Default case, scale config is set to small
           if [[ -z "${scale_config}" ]];then
             scale_config=small
           fi
   
           check_resources $scale_config
   }
   
   check_upgrade_case

4. Run the precheck_migration.sh to determine whether you can run an automatic migration of the common core services or whether you need to configure common core services to run a semi-automatic migration:

   ./precheck_migration.sh 

   Take the appropriate action based on the message returned by the script:

   Messages returned by the script	Migration type	What to do next
   The system is ready for migration. Upgrade your cluster as usual.	Automatic	You are ready to upgrade IBM Software Hub. Important: After you upgrade the services in your environment, ensure that you complete Completing the catalog-api service migration
   Run the following command to increase the CPU and memory.	Automatic	Run the patch command returned by the script. Upgrade IBM Software Hub. Important: After you upgrade the services in your environment, ensure that you complete Completing the catalog-api service migration
   The script returns both of the following messages: Run the following command to increase the CPU and memory. Before you can start the upgrade, you must prepare the system for migration.	Semi-automatic	Run the patch command returned by the script. Run the following command to enable semi-automatic migration: oc patch ccs ccs-cr \ -n ${PROJECT_CPD_INST_OPERANDS} \ --type merge \ --patch '{"spec": {"use_semi_auto_catalog_api_migration": true}}' Upgrade IBM Software Hub. Important: After you upgrade the services in your environment, ensure that you complete Completing the catalog-api service migration