Product Documentation
Abstract
This document describes the general information and diagnostic data needed to start troubleshooting issues related to IBM Business Automation Studio containers, which are included in IBM Cloud Pak for Business Automation. Include the diagnostics retrieved from using this document when you open a case for Business Automation Studio in Cloud Pak for Business Automation.
Content
Overview of Business Automation Studio diagnostic information
General diagnostic information
As needed diagnostic information
Detailed diagnostic collection steps
Use these detailed steps to gather different types of data for BA Studio. When you run the diagnostic commands, run the commands from an empty collection directory to make it easy to package the files. Run the commands from the project or namespace containing BA studio or use the -n <namespace> flag with all oc commands.
Note: oc commands are interchangeable with kubectl.
1: Provide a detailed description of the problem and your environment
- Provided a detailed description of your issue. Include screen captures and re-create steps if possible.
Is it an intermittent or recreatable issue? Has this problem always been an issue or only started after a change occurred?
What is the business impact? Do we need to be aware of any deadlines impacted by the issue? - Provide a reference to the documentation being followed for the failing operation
- Which platform are you using (OpenShift, managed OpenShift, other Kubernetes platform)?
- For Business Automation Studio, what is the database type and version?
2: Gather the configuration information
oc get icp4acluster -oyaml > CP4BAconfig.yaml
oc get content -oyaml > ContentCR.yaml
oc adm must-gather --image=icr.io/cpopen/cpfs/must-gather:latest -- gather -m automationfoundation -n <cloud pak namespace>
oc adm must-gather --image=icr.io/cpopen/cp4ba/icp4a-must-gather:23.0.2 -- gather -m cp4ba –p workflow_authoring -n cp4baNS
Workflow Process Service Authoring:
oc adm must-gather --image=icr.io/cpopen/cp4ba/icp4a-must-gather:23.0.2 -- gather -m cp4ba –p wfps_authoring -n cp4baNS
oc adm must-gather --image=icr.io/cpopen/cp4ba/icp4a-must-gather:23.0.2 -- gather -m cp4ba –p application -n cp4baNS
Automation Document Processing Authoring:
oc adm must-gather --image=icr.io/cpopen/cp4ba/icp4a-must-gather:23.0.2 -- gather -m cp4ba –p document_processing -n cp4baNS
3: Log and Tracing data for WebSphere Liberty
- Edit the icp4acluster CR used by the operator create the bastudio pods.
Modify the traceSpecification property in the bastudio logs section of the yaml and set the following trace string.spec: ... bastudio_configuration: ... logs: trace_specification: '*=info:com.ibm.bpm.rest.*=all:com.ibm.bpmsdk.*=all:com.ibm.bpm.socialbus.*=all'
Update the CR with the new configuration using your preferred method. For example the edit command can be used.oc edit icp4acluster
Note: It can take a large amount of time to recognize the change (length of an operator reconcile) and update the studio configuration. You can grep the log file for traceSpecification to see when the trace settings change.
-
Optional: To have the changes applied immediately, modify the configmap ending with bastudio-overrides-configmap.
This configmap should contain a trace-specification.xml file. Edit the settings of this file to match what was used in the CR file. - Re-create your issue and gather the BA Studio log files. The following command can be used to gather the logs where pod name is one of the BA Studio pods.
oc cp <pod-name>:/logs/application/BAS ./BAS
Note: The logs can also be gathered directly from the associated persistent volume(PV) - Disable the trace by setting traceSpecification back to "*=info" and applying the changes again.
4: Export of your application
5: Collect Operator logs
oc cp $operator_pod_name:/tmp/ansible-operator/runner/ ./operator_logs/
6: Collect Browser data for UI issues
- Network traffic capture export in .har or .saz format. See Collect a HTTP traffic capture with Fiddler or your web browser.
- Export the browser console log.
Open the browser developer tools (can be accessed by pressing F12), and copy the contents of the console tab.
7: Gathering verbose:gc, javacores and heapdumps
- Determine the names of the BA Studio server pods by using the get pods command.
oc get pods | grep bastudio-deployment
-
If dumps need to be generated, you can use the Liberty server dump commands to create them. Use the javadump command to generate javacores for each BAW server pod. Include the option --include=heap or --include=system to generate heapdumps or system core dumps. For example, the following command generates a javacore and heapdump for the pod.
oc exec <podname> -- bash -c "server javadump --include=heap"
Note: If a BA Studio Liberty server JVM crashes, then dumps will be generated as well. -
Provide the dumps by tarring the files in the BA Studio dump persistent volume (bastudio-dump-pvc). This command can be used on any ba studio pod that shares the dump pvc.
oc cp <pod-name>:output/dump ./BAS/dump
Note: You can also get the files by directly accessing the related persistent volume.
Enabling verbose:gc and other JVM dump options.
- Update the CR to include the needed JVM options and point the logs at an appropriate location.
bastudio_configuration jvm_customize_options: -verbose:gc -Xverbosegclog:/logs/application/BAS/verbosegc/verbosegc.%Y%m%d.%H%M%S.%pid.txt,20,10000 -Xdump:stack:events=allocation,filter=#25m
These options enable verbose:gc, send log files to the logging PVC under the BAS/verbosegc directory and enables stack dump for gc events larger than 25MBs. - The operator will rollout the changes. To confirm the change or speed up the process, you can view or edit the icp4adeploy-bas-credential-secret. The jvm.options key in this secret contains the settings. The pods do need to be restarted to pickup the new settings if you change the secret.
- Once enabled logs can be gathered from the logging PV as mentioned in item 3 of this mustgather.
8: Gathering resource registry data
- Get a dump of the resource registry contents. Run this command from one of the resource registry pods.
etcdctl --cacert=/shared/resources/tls/ca-cert.pem --user=root:<root password> --insecure-skip-tls-verify get "" --from-key
- Enable this trace string in addition to any other needed tracing when recreating the issue:
com.ibm.bpm.dbaregistry.*=all: com.ibm.bpm.resourceregistry.*=all: com.ibm.bpm.serviceregistry.*=all: com.ibm.bpm.bas.registry.*=all
What to do next
- Review the log files and traces at the time of the problem to try to determine the source of the problem.
-
Check these locations for known issues:
-
Search in the Cloud Pak for Business Automation Support Page.
-
Review the Business Automation Studio documentation.
-
- Once you completed gathering all the needed information and diagnostics, you can add them to your case. Alternatively, you can upload files to ECURep. For more information, see Enhanced Customer Data Repository (ECuRep) - Overview.
Related Information
Document Location
Worldwide
Was this topic helpful?
Document Information
Modified date:
06 February 2024
UID
ibm11078569