Enabling and managing trace for a deployed integration runtime

You can enable user or service trace on startup of an integration runtime by using the server.conf.yaml file. If you want to capture trace from the earliest possible time, including the period during which the server.conf.yaml file is read, you can instead set an environment variable within the integration runtime custom resource (CR) before you deploy the integration.

To enable trace before you deploy an integration, complete one of the following steps:

• If you want to capture trace from the earliest possible time, including the period during which the server.conf.yaml file is read, set the MQSI_FORCE_DEBUG_TRACING environment variable to a non-null value within the CR that you want to use to create the integration runtime. You can enable only service trace by using the MQSI_FORCE_DEBUG_TRACING environment variable.
  Use the spec.template.spec.containers.env parameter to set the environment variable for use within the runtime container in the deployment as follows:

  spec:
    template:
      spec:
        containers:
          - name: runtime
            env:
              - name: MQSI_FORCE_DEBUG_TRACING
                value: '1'
            resources:
              ...

  While using the App Connect Dashboard to deploy the integration, you can add the spec.template.spec.containers.env settings by switching to the YAML editor while in the Properties view.

  After you create the integration runtime, its tile on the Runtimes page of the App Connect Dashboard displays a status of Ready (trace enabled).

  

  To download the trace information that is captured for the deployed integration, you can run a set of commands as described in Downloading the trace log files.

  Note:

  • When you enable trace by using the MQSI_FORCE_DEBUG_TRACING environment variable, a trace object is not created like it is when you use one of the Start trace menu options on the tile.
  • To stop the trace, you must edit the integration runtime CR to remove the spec.template.spec.containers.env setting. The (trace enabled) notation is removed from the tile when tracing stops. (You might need to refresh the browser tab or window to see this update.) For information about editing the CR, see Editing the settings for a deployed integration runtime. You cannot use the Stop trace menu option on the tile to stop the trace.
• If you require user or service trace on startup, you can deploy an integration runtime with a configuration object of type server.conf.yaml, which defines settings to enable trace.
  1. Before you create the integration runtime, use a server.conf.yaml file to create a configuration object of type server.conf.yaml.
     1. Enable trace in the server.conf.yaml file by using the trace or userTrace properties.

        The following example shows the trace and userTrace properties in a server.conf.yaml file in which user trace is enabled with a debug value.

        ...
        #trace: 'none'               # choose 1 of : none|debug|debugTree|diagnostic|diagnosticTree
        #traceSize: '1G'             # Set the service trace size
        
        #traceNodeLevel: true        # Enable or disable message flow 'Trace nodes'
        
        userTrace: 'debug'           # choose 1 of : none|debug|debugTree
        userTraceSize: '1G'          # Set the user trace size
        ...

     2. Save the file and then use it to create a configuration object of type server.conf.yaml. For more information, see server.conf.yaml type and Configuration reference.
  2. Define the settings for the integration runtime that you want to create and ensure that you specify the name of the configuration object of type server.conf.yaml. To do so, you need to select the configuration object's checkbox in the Configuration view. Then, complete the steps to deploy the integration runtime.
     For more information, see Creating an integration runtime to run your BAR file resources.
  3. From the Runtimes page of the App Connect Dashboard, locate the tile for the integration runtime.

     The status on the tile is displayed as Ready (trace enabled). In the options menu, a Stop trace menu option is also displayed for the type of trace that is enabled, and no Start trace menu option is displayed for the alternative trace type. The following example shows the tile for an integration runtime for which user trace is enabled on startup, and the trace menu options.

     

     To download the trace information that is captured for the deployed integration, see Downloading the user or service trace log files.

     Note:

     • When you enable trace on startup by using a configuration object of type server.conf.yaml, a trace object is not created like it is when you use one of the Start trace menu options on the tile.
     • To stop the trace, you must update the configuration object of type server.conf.yaml to set trace or userTrace to none, or to comment out the properties, as shown in the following example. This configuration update triggers a restart of the integration runtime and the (trace enabled) notation is removed from its tile when you refresh (or reload) the Runtimes page. For information about updating a configuration object, see Managing configuration objects from the Configuration page. You cannot use the Stop trace menu option on the tile to stop the trace.
       

You enable user or service trace on a running integration runtime by starting the trace. You can start a trace only if no other trace is currently enabled.

To start user or service trace for a deployed integration runtime, complete the following steps:

1. From the Runtimes page of the App Connect Dashboard, locate the tile for the relevant integration runtime.
   The integration runtime should be in a Ready state.
2. Open the options menu on the tile. If trace is not yet enabled, you should see the Start service trace and Start user trace options.
3. Click the Start option for the trace that you want to enable, as shown in the following example.
   

   A notification is displayed indicating that tracing has started (for example, Success: Started user trace), and the Ready status on the integration runtime tile changes to Ready (trace enabled).

   

   Tip: The Start action automatically creates a trace object with the same name as the integration runtime. You can view this object from the Red Hat OpenShift web console or CLI, or the CLI for a Kubernetes environment. For more information, see Trace reference.

   
   

   If you open the options menu again on the tile, you should notice that the Start option for the trace that you enabled (for example, user trace) has been replaced by a Stop option. Also note that the Start option for the alternative trace type (for example, service trace) is no longer visible because only one trace can be enabled per integration runtime.

   

   If you want to start an alternative trace, you need to stop the active trace that is currently enabled. For example, if user trace is enabled and you want to switch to service trace, you must stop the user trace and then start the service trace.

   Note: If an integration runtime container is crashing, you should be able to collect trace as normal by setting an environment variable to prevent the container from stopping. For more information, see Collecting trace from an integration server or integration runtime container that is crashing.

After collecting trace information for analysis, you can stop the trace.

To stop an active user or service trace on an integration runtime, complete the following steps:

1. From the Runtimes page of the App Connect Dashboard, locate the tile for the integration runtime.
2. Open the options menu on the tile and locate the Stop option for the active trace.
   For example, if user trace is currently enabled, you should see a Stop user trace option. Similarly, if service trace is enabled, you should see a Stop service trace option.

   

   A notification is displayed indicating that tracing has stopped (for example, Success: Stopped user trace), and the Ready (trace enabled) status on the integration runtime tile reverts to Ready.

   Tip: The Stop action automatically deletes the associated trace object that was created when the trace was started, which means that the object will no longer be visible from the Red Hat OpenShift web console or CLI, or the CLI for a Kubernetes environment. However, the trace log files are retained in the system. Also, note that if you delete an integration runtime, any trace object that references that integration runtime (in spec.integrationRuntimeName) is also automatically deleted. For more information, see Trace reference.

To download the user or service trace log files for an integration runtime, complete the following steps:

1. From the Runtimes page of the App Connect Dashboard, locate the tile for the integration runtime.
2. Open the options menu on the tile and click the relevant Download trace option.
   Depending on your browser, the file might automatically be downloaded to a configured download location or you might be prompted to choose a location.
3. Wait for the download to complete.
   By default, the file is named namespaceName-integrationRuntimeName-traceType-identifier.zip; for example, ace-fiona-ir-fd-toolkit-user-1677008332866.zip or ace-fiona-ir-fd-toolkit-service-1677008332866.zip.
4. Extract the contents of the ZIP file to a preferred location.

   Trace type	Extracted contents
   User trace	For each replica pod, you obtain a user-trace-files.zip file within a directory that is named after the pod; for example, extractedDir/ir-fd-toolkit-ir-5db6584b96-7jcvx/user-trace-files.zip. This ZIP file contains one or more TXT files to which user trace information is written. The TXT files are named in the format integration_server.integrationRuntimeName.userTrace.n.txt; for example, integration_server.ir-fd-toolkit.userTrace.0.txt. The value of n is an integer in the range 0 through 4, enabling trace to be written to five trace log files in a rotational manner. When user trace is started, trace information is written to the file ending in 0.txt. When the 0.txt file becomes full, trace information is then written to the 1.txt file. The process continues until the 4.txt file is full, at which point the existing trace information in 0.txt is overwritten with new information. The process continues in this rotational manner until user trace is stopped.
   Service trace	For each replica pod, you obtain the following files within a directory that is named after the pod (for example, extractedDir/ir-fd-toolkit-ir-5d65848d8b-bpghk): pod-logs.zip: Contains logs from all containers in the pod service-trace-files.zip: Contains the following files: One or more TXT files to which service trace information is written These files are named in the format integration_server.integrationRuntimeName.trace.n.txt; for example, integration_server.ir-fd-toolkit.trace.0.txt. The value of n is an integer in the range 0 through 3, and is used as a numbering mechanism for a series of four files into which the trace log is divided. The exception log (ending in exceptionLog.txt) An env.txt output of environment settings

   The following example shows partial contents of a user trace log file.

   

You can use the reset option to erase the trace information that has been written to an integration runtime's trace log files. You might choose to clear the log before you start a new trace, to ensure that all the logged information is unique to the new trace.

To clear user or service trace information for an integration runtime, complete the following steps:

1. From the Runtimes page of the App Connect Dashboard, locate the tile for the integration runtime.
2. Open the options menu on the tile and click the relevant Reset trace option.
   You should see a notification confirming that the trace has been reset (for example, Success: Reset user trace).