Troubleshooting
It is necessary to collect specific information about your deployments to measure their well-being and to diagnose a problem before you open a support case on Cloud Pak for Business Automation.
Collect specific data about your environment and your Cloud Pak installation before you contact IBM support for assistance with a Cloud Pak for Business Automation issue. Always provide a detailed description of the problem and your environment.
When you run diagnostic commands, run them from an empty directory to package the files more cleanly. Run the commands from the namespace in which you observe the problematic container or component. For more information, see Mustgather: Collecting data to diagnose issues.
Collecting data
- Collect the following information from the scripts directory.
- <location of cert-kubernetes>/scripts/cp4a-script-logs
- <location of cert-kubernetes>/scripts/cpfs/installer_scripts/cp3pt0-deployment/logs
- <location of cert-kubernetes>/scripts/cp4ba-upgrade
- <location of cert-kubernetes>/scripts/.tmp
- <location of cert-kubernetes>/scripts/*.log
- <location of cert-kubernetes>/scripts/*.yaml
- Collect the output from the console where you run the upgrade script.
- Screenshot of the error messages if applicable.
- Run the Cloud Pak for Business Automation Mustgather.
Resolving issues
- MongoDB migration fails when you upgrade from 21.0.3-IF031 or later to 24.0.0 or interim fixes
-
During the upgrade of MongoDB, you might see the following error.
[INFO] Waiting for MongoDB copy to initialize icp-mongodb-0 0/1 Init:1/2 0 10s [INFO] Waiting for MongoDB copy to initialize icp-mongodb-0 0/1 Init:1/2 0 20s [INFO] Waiting for MongoDB copy to initialize icp-mongodb-0 0/1 Init:1/2 0 31s [INFO] Waiting for MongoDB copy to initialize icp-mongodb-0 0/1 Init:1/2 0 41s [INFO] Waiting for MongoDB copy to initialize icp-mongodb-0 0/1 Init:1/2 0 52s [INFO] Waiting for MongoDB copy to initialize icp-mongodb-0 0/1 Init:1/2 0 62s [INFO] Waiting for MongoDB copy to initialize icp-mongodb-0 0/1 Init:1/2 0 73s [INFO] Waiting for MongoDB copy to initialize icp-mongodb-0 0/1 Init:1/2 0 83s [INFO] Waiting for MongoDB copy to initialize icp-mongodb-0 0/1 Init:1/2 0 94s [INFO] Waiting for MongoDB copy to initialize Error from server (NotFound): pods "icp-mongodb-0" not found [✘] Error occurred in function deploymongocopy at line 1842 [✗] Failed to execute command: /root/cert-kube-2400/cert-kubernetes/scripts/cpfs/installer_scripts/preload_data.sh --original-cs-ns ibm-common-services --services-ns test-ns --yq "/root/cert-kube-2400/cert-kubernetes/scripts/cpfs/yq/amd64/yq" [ATTENTION]: You can run follow command to try upgrade again after fix migration issue of IBM Cloud Pak foundational services. # ./cp4a-deployment.sh -m upgradeOperator -n combank1 --cpfs-upgrade-mode <migration mode> --original-cp4ba-csv-ver <cp4ba-csv-version-before-upgrade>(B[m Usage: --cpfs-upgrade-mode : The migration mode for IBM Cloud Pak foundational services, the valid values [shared2shared/shared2dedicated/dedicated2dedicated] --original-cp4ba-csv-ver: The version of csv for CP4BA operator before upgrade, the example value [21.3.31] for 21.0.3-IF031/[22.2.6] for 22.0.2-IF006/[23.2.3] for 23.0.2-IF006 Example command: # ./cp4a-deployment.sh -m upgradeOperator -n test-ns --cpfs-upgrade-mode dedicated2dedicated --original-cp4ba-csv-ver 21.3.31
Solution: Run thepreload_data.sh
script with the--clean
option. For more information, see Preload data. After you complete running thepreload_data.sh
, run the command that is listed in the upgrade script output../cp4a-deployment.sh -m upgradeOperator -n test-ns --cpfs-upgrade-mode dedicated2dedicated --original-cp4ba-csv-ver 21.3.31
- Upgrade blocked due to missing
ibm-cp4ba-common-config
ConfigMap -
If the Not found "ibm-cp4ba-common-config" configMap error message is encountered when the upgrade script runs, the
ibm-cp4ba-common-config
ConfigMap is missing from the namespace for the CP4BA deployment.To resolve the issue, follow the steps in either Option 1 or Option 2.
Option 1: Create the missing ConfigMap manually
Use the following steps to re-create the ConfigMap.
- Find the namespace where the
CommonService
CR is installed by running the following command. The namespace of theCommonService
deployment is where all the Cloud Pak foundational services and CP4BA operators are installed.oc get CommonService -A
Note: If you have multiple deployments, make sure that you locate the namespace that you are upgrading before you complete the next steps. - Run the following commands to retrieve the values needed to create the
ibm-cp4ba-common-config
ConfigMap.oc get CommonService -o yaml | grep 'operatorNamespace:' | head -n 1 | awk '{print $2}' oc get CommonService -o yaml | grep 'servicesNamespace:' | head -n 1 | awk '{print $2}'
- Create the
ibm-cp4ba-common-config
ConfigMap in the targetCommonService
namespace.kind: ConfigMap apiVersion: v1 metadata: name: ibm-cp4ba-common-config namespace: <namespace> data: operators_namespace: "<operator-namespace>" services_namespace: "<service-namespace>"
Where the value of <namespace> and <operator-namespace> is provided by the first command in the previous step. The value of <service-namespace> is provided by the second command.
- If you created a YAML file in the previous step, for example
ibm-cp4ba-common-config.yaml, then run the following command to apply the
ConfigMap to the
CommonService
namespace from the command window.oc apply -f ibm-cp4ba-common-config.yaml -n <namespace>
If you prefer to use the OCP console, then apply the ConfigMap manually.
- After you created the
ibm-cp4ba-common-config
ConfigMap, return to upgrading the deployment by running the cp4a-deployment.sh script inupgradeOperator
mode.
Option 2: Run the cp4a-clusteradmin-setup.sh script to create the missing ConfigMap
- Run the cp4a-clusteradmin-setup.sh script from the local directory where
you downloaded the cert-kubernetes repository, and follow the prompts in the
command window.
cd cert-kubernetes/scripts ./cp4a-clusteradmin-setup.sh -fix_configmap
- After you created the
ibm-cp4ba-common-config
ConfigMap, return to upgrading the deployment by running the cp4a-deployment.sh script inupgradeOperator
mode.
- Find the namespace where the
Troubleshooting capabilities
The custom resource can be configured to enable and disable specific logging parameters, log levels, log formats, and where these logs are stored for the various capabilities. If you need more information about specific Cloud Pak capabilities, go to the relevant troubleshooting topics.