Enhancing logging with customized log messages

In IBM® App Connect on IBM Cloud®, logs are provided in a built-in log viewer to help you monitor the behavior of your flows and diagnose errors. Your App Connect logs can also be forwarded to an IBM Log Analysis with LogDNA service instance that provides the capability to aggregate, monitor, and manage event logs from multiple IBM Cloud sources in a centralized location.

You can enhance logging by defining your own custom messages that you want App Connect to send to the logging service. You can also define custom fields for specific content that you want to log. This capability offers some control over what is written to the logs, so you can display information that's important to you and locate your log entries more easily.

You can configure logging while creating or editing a flow by adding one or more Log nodes with a variety of log levels, custom messages, and custom fields.

You can can add a Log node at any stage of the flow, after the event node that acts as a trigger. The Log node is available from the Toolbox tab in the flow editor.

Tip: You can also use a Log node to test your flow. For example, you can temporarily add a Log node to your flow to record some output from a previous node so that you can verify whether that node is correctly configured to provide the expected output.

Configuring logging

To add and configure a Log node within a flow, complete the following steps:

  1. At the stage in the flow where you want to capture relevant data from previous nodes or provide contextual information that should be written to the logs, click (+) and then from the Toolbox tab, click Log.
    Figure 1. Selecting the Log node from the Toolbox
    Selecting the Log node from the Toolbox

    (Click image to view full size.)

    In the preceding example image, a ServiceNow "New incident" event is defined as the trigger for the flow, followed by a Set variable node with user-defined transactionId and priority data properties and mapped values. (The transactionId property is configured to resolve to a unique auto-generated value, and the priority value is calculated from the Impact and Urgency values recorded for the incident.)

    • Example of the Set variable node mappings

    In our example, we want to configure the Log node so that whenever a new incident is created, a message is written to the logs to indicate that a new incident is being processed. We'd also like to capture the transaction ID and priority for the incident.

  2. In the Log fields, configure the message that you want to pass to your App Connect logs:
    • Log level: Specify one of the following values:
      Info
      Log an informational message at that step in the flow.
      Error
      Log information about an unexpected situation or error in the flow. Errors are generally emitted when an operation fails, and are depicted with a warning icon Flow error icon on the flow's tile on the Dashboard. You can also choose to log a "custom" error about something significant that might not necessarily result in an HTTP error code being returned.
      Debug
      Log detailed information for debugging purposes.
      Note: Debug messages are written to the logs only if debug logging is enabled on the flow. For information about how to enable debug logging, see Viewing App Connect logs in the log viewer.
      Custom
      Specify a custom value (such as a JSONata mapping) that should resolve to Info, Error, or Debug. If left as Custom, the log level defaults to Info.
    • Message detail: Specify the message text that should be displayed in the Message column for that log entry.

      In our example, we've defined a log level of Info, together with a custom message to indicate the new incident is being processed.

      Figure 2. Initial fields in the Log node
      Initial fields in the Log node

      (Click image to view full size.)

  3. If you want to define your own fields to include in the log entry, add these custom fields as data properties with defined values:
    1. To define a data property, click Add property under Custom message fields to display the property editor. Then enter a property name and select its data type.
    2. Use Add property to define additional data properties.

      In the following example, we've defined two data properties that will contain a priority and a transaction ID for the incident.

      Figure 3. Example of defined properties under 'Custom message fields'
      Example of defined properties under 'Custom message fields'

      (Click image to view full size.)

      Note: If you define a data property as an object or array of objects with nested properties, the parent property will be displayed as a field in the log viewer, and the name/value pairs for the nested properties will be shown as the field value, in JSON format.
      • Example of nested custom fields in the Log node and log viewer
  • To expose the properties as fields, click Edit mappings. You can now define values for these fields.
    Figure 4. Custom properties exposed as fields
    Custom properties exposed as fields

    (Click image to view full size.)

  • For each field, specify a value by mapping to fields from previous nodes in the flow or by entering text. In our example, we've mapped to the priority and transactionId data properties that we set up in the Set variable node because we want their values to be written to the logs.
    Figure 5.
    Custom properties exposed as fields with mapped values

    (Click image to view full size.)

    Note:
    • You can add or remove properties by clicking Edit properties to switch back to the property editor.
    • If you want to update the name or data type of a defined property, ensure that the associated field is blank before you click Edit properties. (In the property editor, properties with defined values are grayed out and cannot be changed.)

You can complete your flow in the usual way by adding actions, additional Log nodes, or other toolbox utilities that you need. When the flow is triggered, you can view the logs to check the information that's recorded.

Viewing your custom messages and fields in the logs

Open the Logs view to see the messages that are logged for your flow (click the Logs icon Logs icon). The built-in log viewer opens to show the event messages that are logged for all your running flows, including any custom messages that you defined in the Log node.

Tip: If error messages were emitted for your flow (for example, if a retrieve action returned no items, or an action has dependencies that weren't met), you can also access the log viewer to view those errors and any custom messages that are specific to your flow. While the flow is running, click the warning icon Flow error icon on the flow's tile on the Dashboard to view the related errors, and then use the View logs option to see flow-specific messages in the log viewer.
  • Example of an error icon on a flow tile and corresponding messages

    (Click image to view full size.)

The following example shows the log viewer with a filter that shows log entries for our specific flow. The custom text (Processing new ServiceNow incident), which we defined earlier, is displayed in the Message column of the highlighted row.

Figure 6. Log viewer with a custom message from the Log node
Log viewer with a custom message from the Log node

(Click image to view full size.)

If you expand the log entry for the custom message by clicking the arrow to the left of the event time, you'll see additional metadata about the event, including the custom fields that you defined and their generated values. Notice that these fields are prefixed with the text user_ to differentiate them from the standard fields (for example, user_incident_priority and user_transaction_id).

For more information about monitoring and managing messages in the log viewer, see Viewing App Connect logs in the log viewer.

If you've configured an IBM Log Analysis with LogDNA instance to receive service logs from App Connect, custom messages from your Log nodes will also be forwarded to this instance. You'll need to directly access your IBM Log Analysis with LogDNA instance in IBM Cloud to see the log entries in the web UI. Here's an example of what the custom message from the preceding image looks like in the IBM Log Analysis with LogDNA web UI. The view is filtered to show only the log entries for our specific flow.

Figure 7. LogDNA log entry for a custom message from the Log node
LogDNA log entry for a custom message from the Log node

For more information about how to configure an IBM Log Analysis with LogDNA instance to receive logs, launch the instance, and monitor and manage log messages in its web UI, see Monitoring and managing App Connect logs in IBM Log Analysis with LogDNA