Customizing data with a gateway script or set-variable operation

Customize your API event records on the gateway before they are sent to your analytics subsystem.

Before you begin

Ensure that you are familiar with the API Manager UI, and working with API definitions.
Note: This feature is not available with DataPower® Gateway (v5 compatible).

About this task

During API execution, the activity data is stored in the log context variable, which populates the API event record on completion of the API execution. For more information about log context variables, see API activity logging context variables.

The analytics subsystem stores analytics data as individual API event records. The API event record fields are documented here: API event record field reference. It is possible to customize the API event records for specific APIs. You can modify the contents of existing API event fields, and add new custom fields by using a gateway script or the set-variable operation.
Note: If you want to make updates to all incoming API event data, rather than specific APIs, you can use ingestion filters instead.

Procedure

  1. In the API Manager UI, click the Develop APIs and products tile, select the APIs tab, click Add, select API.
  2. Select New OpenAPI, give the API the title of Hello World, and click Next. Click Next again to accept the security defaults, and then click Edit API.
  3. Select the YAML source view OpenAPI Source icon. Replace the generated YAML with the following YAML and click Save:
    swagger: '2.0'
    info:
      title: Hello World
      x-ibm-name: hello-world-rest
      version: 1.0.0
    schemes:
      - https
    basePath: /hello-world
    securityDefinitions:
      clientID:
        type: apiKey
        in: header
        name: X-IBM-Client-Id
    
    x-ibm-configuration:
      phase: realized
      testable: true
      enforced: true
      properties:
        target-url:
          value: http://example.com/operation-name
          description: The URL of the target service
          encoded: false
      cors:
        enabled: true
      activity-log:
        enabled: true
        error-content: header
        success-content: activity
      application-authentication:
        certificate: false
      assembly:
        execute:
          - map:
              version: 2.0.0
              title: map
              inputs:
                name:
                  schema:
                    type: string
                  variable: request.parameters.name
              outputs:
                response:
                  schema:
                    $ref: '#/definitions/Result'
                  variable: message.body
              actions:
                - set: response.hello
                  from: name
                  value: '''Hello '' +$(name)'     
        catch: []
        finally: []
      gateway: datapower-api-gateway
      categories:
        - /tests
      type: rest
    definitions:
      Result:
        type: object
        additionalProperties: false
        properties:
          hello:
            type: string
    paths:
      /hello:
        get:
          parameters:
            - name: name
              in: query
              required: false
              type: string
          responses:
            '200':
              description: OK Response
              schema:
                $ref: '#/definitions/Result'
        parameters: []
    
  4. Click the Form icon Form icon to switch to the Form view and select the Gateway tab.
  5. Ensure that Policies is selected.
  6. On the canvas, click the plus icon Plus sign inside a circle at the last position in the flow. The palette listing the available elements to build the assembly flow appears.
  7. From the palette, add the Log policy element onto the canvas by clicking it. Set the log policy Mode drop-down to Gather-only. This step is so that the log context variable is populated, for use in the next step.
    Policies page showing the property sheet of the Log policy on the right side
  8. Modify API event data or add custom data either with a gateway script or by using the set-variable operation.
    Gateway script method
    Add a gateway script policy to the assembly flow and paste in the following script code:
    var logs = context.get('log');
      if (logs) {
        if (!logs.custom_data) {  
          logs.custom_data = {};
        }
        logs.custom_data.newAPIEventDataField = 'newData';
        logs.product_title = 'newProductTitle';
        context.set('log', logs);
      }
    Policies page showing the property sheet of the GatewayScript element on the right side In this example, the new field that is added to the API event record is called newAPIEventDataField and its value is set to newData. The value of the existing field product_title is changed to newProductTitle.
    Set-variable method
    If you cannot or do not want to use a gateway script, then you can use a set-variable policy instead. Add a set-variable policy to the assembly flow, and set it as follows: Policies page showing the property sheet of the Set Variable element on the right sideIn this example, the new field that is added to the API event record is called newAPIEventDataField and its value is set to newData
    The YAML code for set-variable is:
          - set-variable:
              version: 2.0.0
              title: set-variable
              actions:
                - set: log.custom_data.newAPIEventDataField
                  value: newData
                  type: string
                - set: log.product_title
                  value: newProductTitle
                  type: string
    Warning: context.log.custom_data supports the addition of primitive typed values such as strings and integers, no limit is enforced on how many you can add, but adding too many does impact performance. Add data only to the context.log.custom_data object. Do not add data to context.log.

    The field types cannot be changed. If you attempt to assign a value of the wrong type to a field, then the event record is lost. For example, if you define a custom field as an integer, and a particular API call results in a string being assigned to this variable, then the analytics database does not store the event record for that API call. If your APIs might return different types, then define the new custom_data type as a string and ensure that everything you assign to it is converted to a string before it is assigned.

  9. Publish your Hello World API to a catalog.
  10. Make a test call to the API and verify that the new custom data is set.
    1. From the Analytics view, select Discover and click the API event record corresponding to your test call:
    2. Scroll down the page to view the API event record payload. Verify that your custom data field is present:Custom data field on the API event record payload
  11. You can use filters to view only analytics data that has your new custom field set.
    Analytics data filter
  12. To use new data with REST API calls, use the custom_data query parameter:
     /cloud/events?custom_data[newAPIEventDataField]=newData