Gathering post-mortem logs

The generate_postmortem.sh script gathers all logs for troubleshooting and diagnostics.

About this task

When you contact IBM Support, API Connect pod logs and associated Kubernetes or OpenShift environment data are normally required to help in diagnostics. The generate_postmortem.sh script gathers all the required logs and data.

The logs that are gathered by the generate_postmortem.sh script include the logs of the current running API Connect containers. The container logs have a maximum size, and older log messages are not retained when the maximum size is reached. To ensure that the logs gathered cover the time when the problem occurred, it is recommended to reproduce the problem and then immediately gather the logs. Also, to ensure that you have logs of previous failed containers, the recommended best practice is to offboard your logs to a remote server for long-term storage. Use ELK, or another logging infrastructure, to gather logs that include terminated pods and processes. For more information, see https://www.elastic.co/elk-stack

Procedure

  1. Download the v10 generate_postmortem.sh script from https://github.com/ibm-apiconnect/v10-postmortem.
  2. Run the script in your Kubernetes or OpenShift environment, following the instructions on https://github.com/ibm-apiconnect/v10-postmortem.
  3. Upload the generated output file with your IBM support case.

What to do next

If the logs do not cover far enough back in time, or conversely if they are too large, adjust the log retention size. For further information, see: https://docs.docker.com/config/containers/logging/json-file/.

If you are not able to run the generate_postmortem.sh script, the API Connect container logs can be gathered manually with the following command:
kubectl logs -n <namespace> <pod name> [-c <container name>]
Redirect the output of the logs command to a file that you can upload to the support case, for example:
kubectl logs -n apic-ns apic-apim-77fdf47c55-vgnr2 > apimpod.log