Troubleshooting

Learn how to collect logs that can help you troubleshoot issues related to Business Teams Service (BTS).

To enable diagnostic traces for the BTS Server, complete the following steps.
1. Use the oc command line tool to edit the BTS Custom Resource:
```
oc edit bts cp4ba-bts -n bts-namespace
```
  where the bts-namespace should be replaced with the name of the namespace where the BTS Custom Resource is installed.
2. Change the traceSpec property under spec section and save the changes:
```
...
spec:
  traceSpec: '*=info:com.ibm.dba.ums.*=all'
...
```
  Tracing in the BTS Server is enabled, and the changes are applied by the BTS operator in the next reconcile.
To enable diagnostic traces for the BTS Operator, create the following ConfigMap in the namespace where the BTS Custom Resource resides:

apiVersion: v1
kind: ConfigMap
metadata:
  name: ibm-bts-operator-config
  namespace: bts-namespace
data:
  # enables operator tracing in general
  traceEnabled: "true"
  # verbose YAML output (only if trace is enabled)
  debugYaml: "true"

The BTS Operator will then output diagnostic trace to its stdout.

To collect logs of your environment, run the following command:

Note: The MustGather tool image location is changed. For more information, see Using MustGather diagnostics through the command line.
```
  oc adm must-gather --image=icr.io/cpopen/cpfs/must-gather:latest
```
To collect logs for BTS only, copy the following snippet in a shell script and make it executable:

Note If the IBM Cloud Pak foundational services namespace is other than ibm-common-services, call the script with this namespace as parameter.

#!/bin/bash

commmon_service_namespace="ibm-common-services"
if [[ $1 != "" ]]
then
  commmon_service_namespace=$1
fi

echo "Check if common services namespace $commmon_service_namespace exists"
nsStatus=$(kubectl get ns $commmon_service_namespace -o jsonpath="{.status.phase}")
if [[ $nsStatus == "Active" ]]
then
  echo "Namespace $commmon_service_namespace exists"
else
  echo "ERROR: Namespace $commmon_service_namespace does not exist. Please provide the correct common services namespace."
  exit 1;
fi

DIR=$(mktemp -d)

#Collecting the description, configuration, logs and referenced secrets of BTS pods
pods="$(kubectl get pod --selector='app.kubernetes.io/component=ibm-bts,role=application-pod' -o custom-columns=NAME:.metadata.name --no-headers)"

for pod in $pods
 do
   echo "Gathering description, configuration and log for $pod"
   kubectl describe pod $pod > $DIR/$pod-description.txt
   kubectl get pod $pod -o yaml > $DIR/$pod-configuration.txt
   echo "Secrets referenced in $pod:" >> $DIR/referencedSecrets.txt
   kubectl get pods $pod -o json |grep secretName >> $DIR/referencedSecrets.txt
   mkdir $DIR/$pod-logs
   kubectl cp $pod:/logs/UMS/$pod/  $DIR/$pod-logs
 done

operatorpod="$(kubectl get pod -n $commmon_service_namespace  -o custom-columns=NAME:.metadata.name --no-headers | grep ibm-bts-operator)" 

echo "Gathering description, configuration and log for $operatorpod" in namespace $commmon_service_namespace
kubectl describe pod $operatorpod -n $commmon_service_namespace > $DIR/$operatorpod-description.txt
kubectl get pod $operatorpod -n $commmon_service_namespace -o yaml > $DIR/$operatorpod-configuration.txt
kubectl logs $operatorpod -n $commmon_service_namespace>  $DIR/$operatorpod.log


 echo "Zip the must-gather directory"
 outputdir=$(pwd)
 (cd $DIR; zip -r $outputdir/bts-must-gather-$(date +"%y-%m-%d_%T").zip ./*)
 rm -rf $DIR

Run the script. Attach the generated .zip file to your support ticket.

To collect CPU or memory profile data for BTS operator, enable the profiling of the BTS operator pod:

$ oc edit subscription ibm-bts-operator

Add the following prarmeters in the config: section to enable profiling of the BTS-Operator pod:

spec:
  channel: alpha
  config:
    env:
    - name: ENABLE_PROFILING_SERVER
      value: "true"
  installPlanApproval: Automatic
  name: ibm-bts-operator
  source: ibm-bts-operator
  sourceNamespace: ibm-bts-operator
  startingCSV: ibm-bts-operator.v3.34.0

To enable connections to the port, you can forward the port to your local machine:

$ oc port-forward pod/ibm-bts-operator-controller-manager-6887d97c8c-f4xqv 8082

Get the profiling data with the following command:

$ curl -s "http://127.0.0.1:8082/debug/pprof/heap" > ~/logs/heap-profile.out
$ curl -s "http://127.0.0.1:8082/debug/pprof/profile" > ~/logs/cpu-profile.out

For more information on the endpoints, see pprof.