Troubleshooting
The must-gather tool gathers information about IBM Cloud Pak® for Integration and the cluster where it is deployed. The information that is collected is required by IBM Support for troubleshooting user issues.
Before you open a case with IBM Support, get the logs and other important information by using the must-gather tool.
Gathering diagnostic information
To help IBM Support troubleshoot any issues with your Cloud Pak for Integration installation, gather:
To get workarounds for known limitations, see Known limitations.
Before you begin
Meet the following prerequisites:
You have admin permissions, which are required to run the must-gather tool.
The OpenShift Container Platform CLI (
oc
) is installed. For more information, see Installing the OpenShift CLI.The OpenShift Container Platform CLI (
oc
) is installed. For more information, see Installing the OpenShift CLI.You have access to the must-gather image. For more information, see Using MustGather diagnostics through the command line for cluster admin in the IBM Cloud Paks documentation.
Gathering Cloud Pak for Integration information
Running the must-gather tool as an administrator
When you run the must-gather tool with admin permissions, the tool automatically finds namespaces where instances are installed and collects all the required logs. It also collect logs from any namespace with the prefix openshift-
in the name.
If you are installing Cloud Pak for Integration in an air-gapped or disconnected environment, skip to the "Air-gapped and disconnected environments" section.
Log in to the cluster:
oc login
Go to the directory where you want to store the must-gather data.
Collect the must-gather information. Make sure you include the quotation marks in the command.
To collect Cloud Pak for Integration must-gather only, run:
oc adm must-gather --image=icr.io/cpopen/cpfs/must-gather:latest -- gather -m cp4i --params "'-c cp4i'"
To collect Cloud Pak for Integration must-gather with a version table, run:
oc adm must-gather --image=icr.io/cpopen/cpfs/must-gather:latest -- gather -m cp4i --params "'-c cp4i -v True'"
To get additional options (help), run:
oc adm must-gather --image=icr.io/cpopen/cpfs/must-gather:latest -- gather -m cp4i --params "'-h'" options: -h, --help Show this help message and exit -n NAMESPACE, --namespace NAMESPACE Namespace you want to collect the logs for, only works with -c appconnect -p PODNAME, --podname PODNAME Pod name you want to collect logs for, only works with -c appconnect -c COMPONENT, --component COMPONENT Component you want to collect logs for. For ACE = appconnect, CP4I = cp4i -v VERSION, --version VERSION To get a table of all the versions installed, set -v True --previous_pod_logs, PREVIOUS POD LOGS To collect pod logs for terminated pods, set --previous_pod_logs True -ne, --namespaced_events NAMESPACED EVENTS To collect events per namespace, set -ne True
Running the must-gather tool as a non-administrator user
If you do not have admin permissions, you can collect logs only from the namespaces where you have user permissions.
Log in to the cluster:
oc login
Run the following command to create a directory called
mustgather
(where you will store the logs that are collected) and to apply permissions so that logs can be written to the directory. This directory is mounted when you run the command for the must-gather tool in the last step.mkdir -p mustgather chmod 777 mustgather Change the dir to mustgather folder cd mustgather
Collect the logs. For the value of
namespaces
, enter a list of comma-separated namespaces (to which you have permissions) that you want to collect logs from.image_name=icr.io/cpopen/cpfs/must-gather:latest namespaces=<my_namespaces>
For example:
namespaces=navigator-ns,ibm-common-services,apic-ns,ace-dashboard-ns,ace-integrationserver-ns
Now run the must-gather tool. Make sure that you include the quotation marks in the command.
podman run --platform linux/amd64 --pull=always -v ${KUBECONFIG:-~/.kube/config}:/kube/config -v $(pwd):/tmp/must-gather \ --env MYENV=NONE --env KUBECONFIG=/kube/config \ --env MUSTGATHERPATH=/tmp/must-gather \ ${image_name} gather -m cp4i --params "-c cp4i --non_admin True -n ${namespaces}"
Air-gapped and disconnected environments
This section applies to installations in air-gapped and disconnected environments only.
You can use the must-gather tool in an air-gapped environment because when you mirrored the Cloud Pak foundational services images (using the procedure in Mirroring images with a bastion host or Mirroring images with a portable compute or storage device (file system)), the must-gather image was mirrored to your local registry.
Set the registry where your local images are mirrored:
export LOCAL_REGISTRY=<YOUR_LOCAL_REPOSITORY>
Collect the must-gather information. Make sure you include the quotation marks in the command.
To collect Cloud Pak for Integration must-gather only, run:
oc adm must-gather --image=${LOCAL_REGISTRY}/cpopen/cpfs/must-gather:latest -- gather -m cp4i --params "'-c cp4i'"
To collect Cloud Pak for Integration must-gather with a version table, run:
oc adm must-gather --image=${LOCAL_REGISTRY}/cpopen/cpfs/must-gather:latest -- gather -m cp4i --params "'-c cp4i -v True'"
To get additional options (help), run:
oc adm must-gather --image=${LOCAL_REGISTRY}/cpopen/cpfs/must-gather:latest -- gather -m cp4i --params "'-h'" options: -h, --help Show this help message and exit -n NAMESPACE, --namespace NAMESPACE Namespace you want to collect the logs for, only works with -c appconnect -p PODNAME, --podname PODNAME Pod name you want to collect logs for, only works with -c appconnect -c COMPONENT, --component COMPONENT Component you want to collect logs for. For ACE = appconnect, CP4I = cp4i -v VERSION, --version VERSION To get a table of all the versions installed, set -v True --previous_pod_logs, PREVIOUS POD LOGS To collect pod logs for terminated pods, set --previous_pod_logs True -ne, --namespaced_events NAMESPACED EVENTS To collect events per namespace, set -ne True
To confirm that you have mirrored the latest available version of the must-gather tool, run:
skopeo list-tags docker://[LOCAL_REGISTRY:5000]/cpopen/cpfs/must-gather { "Repository": "${LOCAL_REGISTRY}/cpopen/cpfs/must-gather", "Tags": [ "4.5.16" ..... "4.6.7", "4.6.8", "4.6.9", "latest" ] }
Information that is collected
The must-gather tool collects information about the following resources:
Certificate
Clients
ConfigMaps
Cp4iServiceBindings
Cloud Pak CustomResources
Deployments
InstallPlan
Jobs
OperandRequests
PersistentVolumeClaims
Pod-Description
Pod-Logs
ReplicaSets
Routes
Services
StatefulSets
Subscriptions
CatalogSources
ServiceAccounts
Events
OperandBindInfo
OperandRegistries
OperandConfigs
Issuer
It collects information from any namespace where operators and instances are installed, as well as the following namespaces:
ibm-common-services
openshift-marketplace
openshift-operator-lifecycle-manager
openshift-operators
openshift-ingress
kube-public
The must-gather tool collects only the name of the secrets from the listed namespaces. It doesn't collect the data in those secrets.
Gathering Cloud Pak foundational services logs only
The script in the previous section, Gathering extended Cloud Pak For Integration information, also returns the pod logs for Cloud Pak foundational services.
However, if you need to work directly with the Cloud Pak foundational services support team, see Collecting Cloud Pak Cluster and Common services for problem determination in the Support documentation. This page describes how to collect logs that are needed by the team that investigates your issue.
Gathering OpenShift Container Platform logs
OpenShift has its own must-gather tool. To gather debugging information about your cluster, see Gathering data about your cluster).
Gathering App Connect logs
For information on collecting must-gather information that is specific to App Connect, see Gathering diagnostic information.
Known limitations and workarounds
To get specific information about troubleshooting known issues in Cloud Pak for Integration, see Known limitations.