IBM Support

Mustgather: Collecting data to diagnose issues with IBM Cloud Pak for Business Automation

Troubleshooting


Problem

This document describes the general information and diagnostic data needed to start troubleshooting issues related to container components included in IBM Cloud Pak for Business Automation (CP4BA). When you open a case for the Cloud Pak for Business Automation, include the diagnostics retrieved from using this document.
Note: Previously called Cloud Pak for Automation

Resolving The Problem

When you contact support for assistance with a Cloud Pak for Business Automation issue, collect the following troubleshooting data.

General Diagnostic Information
These items are the general diagnostics, which are useful in most situations regardless of component.  When you run the diagnostic commands, run them from an empty collection directory to make it easy to package the files. Run the commands from the project or namespace containing Cloud Pak for Business Automation or use the -n <namespace> flag with all oc commands.
Note: oc commands are interchangeable with kubectl.

1: Provide a detailed description of the problem and your environment

  • Provided a detailed description of your issue. Include screen captures and re-create steps if possible.
    Is it an intermittent or recreatable issue? Has this problem always been an issue or is it an issue that started only after a change occurred?
    What is the business impact? Do we need to be aware of any deadlines impacted by the issue?
    Provide details on which component of the Cloud Pak has issues.
  • Provide a reference to the documentation being followed for the failing operation.
  • Is this environment development, test, or production?
  • Which platform setup are you using (OpenShift, OpenShift on IBM Cloud Public, other Kubernetes platform)?
  • What is the database type and version?
     

2: Gather configuration information

Option 1: Option 1 gets all the information about resources in the namespace. It additionally gets the output of oc logs for each pod.
oc get icp4acluster -o yaml > CP4BAconfig.yaml

oc adm must-gather --image=icr.io/cpopen/cpfs/must-gather:latest -- gather -m automationfoundation -n <cloud pak namespace>
The -n parameter is required and must be a single namespace.  If using an air gap setup, you should ensure you have pushed the latest version of the must-gather image into your local repository. The command requires cluster admin access to execute.
Generally, this collection takes 5-10 minutes and produces a 25-50MB gzip file.

If you are having issues with foundational services in the ibm-common-services namespaces, then it is recommended to also get the collection for that namespace as well:
oc adm must-gather --image=icr.io/cpopen/cpfs/must-gather:latest -- gather -m automationfoundation -n ibm-common-services
Option 2: Option 2 provides some basic configuration data if you aren't able to gather option 1. It gets far less data and depending on the problem we may have to request more configuration data as the problem progresses.
  • Provide the OpenShift and Kubernetes version information:
    oc version > version.txt
    kubectl version >> version.txt
  • Provide the operator version information:
    oc get csv > operatorInfo.txt
    oc get csv -n ibm-common-services >> operatorInfo.txt
  • Provide the Custom Resource(CR) .yaml file used by the operator to configure the environment
    oc get icp4acluster -o yaml > config.yaml
  • If you have the needed permissions, collection information about the nodes.
    oc get nodes -o wide > nodes.txt
    oc get mcp > mcp.txt
  • Collect information about the pod statuses
    oc get pods > pods.txt
  • Collect information about the pod containers
    oc get pods -o jsonpath="{..image}" > containerInfo.txt
  • On OpenShift gather route configuration
    oc get route > routes.txt
  • Collect the defined secrets
    oc get secrets > secrets.txt
  • Collect the defined persistent volume claims
    oc get pvc > pvcs.txt
  • For installation or upgrade problems, get the job information:
    oc get jobs > jobs.txt
  • Collect the description and log of any pod you are having issues with:
    oc describe pod <pod-name> > describe-<podname>.txt
    oc logs <podname> > log-<podname>.log

3: Collect Operator logs

If you are having issues during the deployment by the operator, then collect the operator logs:
  • Get the pod logs with this command:
    oc logs $operator_pod_name > $operator_pod_name.log
  • If the issue is with CP4BA's cp4a, foundation, or content operators, get the logs from the most recent completed reconcile:
    oc cp $operator_pod_name:/logs/$operator_pod_name/ansible-operator/runner/ ./operator_logs/
Where $operator_pod_name is the name of the operator pod you are concerned with. For more details, see the installation troubleshooting page.
 

4: Collect Browser data for UI issues

 
For console or web application usage issues, capture the following browser data:

5: Collect data when reporting a security vulnerability

If you are reporting a possible security vulnerability or asking about an existing CVE, see IBM Cloud Pak for Business Automation Security Vulnerability Policy.
This document includes details on our general policies and what to provide when reporting an issue.

Component-Specific Diagnostics
For issues related to a particular Cloud Pak container component, we recommend reviewing the MustGather or troubleshooting page for that component.

What to do next

  1. Review the diagnostic at the time of the problem to try to determine the source of the problem.
     
  2. Check these locations for known issues:
  3. Once you complete gathering all the needed information and diagnostics, you can add them to your case. Alternatively, you can upload files to ECURep. For more information, see Enhanced Customer Data Repository (ECuRep) - Overview.

Document Location

Worldwide

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS7JTW","label":"IBM Digital Business Automation"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
01 November 2022

UID

ibm16120897