Configuring offload on Kubernetes and Red Hat OpenShift

Configure your analytics subsystem to offload API event data to a third-party system.

Procedure

  1. Edit your analytics CR:
    kubectl edit a7s
    Note: If you are using the OpenShift® top-level CR deployment, edit your APIConnectCluster CR instead. Update the spec.analytics section.
  2. Set spec.external.offload to true, and add the offload plug-in configuration:
    The following example shows an HTTP output plug-in:
    spec:
      external:
        offload:
          enabled: true
          output: |
            http {
              url => "https://example.com"
              http_method => "post"
              ssl_verification_mode => "none"
              id => "API Connect analytics event data"
            }

    In this example, the URL points to an HTTPS endpoint, but verification of the endpoint's server certificate is disabled by setting ssl_verification_mode to "none". If you want your analytics subsystem to verify the endpoint server certificate, then see Configuring offload server certificates on Kubernetes and Red Hat OpenShift.

    The id property is not mandatory, but it is recommended.

    For more information about the HTTP output plug-in configuration, see the Logstash HTTP plugin documentation.

  3. Optional: Add a filter after the output plug-in configuration to modify data before it is sent to the offload target.
    The following example filter removes the query_string field:
    spec:
      external:
        offload:
          enabled: true
          output: |
            http {
              ...
            }
          filter: |
            ruby {
              remove_field => ["query_string"]
            }
    For information about modifying offloaded data, see Customizing data with ingestion filters.
  4. Optional: To configure a second offload target, create a new spec.external.offload2 section and add the configuration for this second target to this section.
    If you have three or more offload targets, then you can put multiple configurations in spec.external.offload2. For example:
    spec:
      external:
       offload:
          enabled: true
          output: |
            http {
              url => "https://target1.com"
              ...
            }
        offload2:
          enabled: true
          output: |
            http {
              url => "https://target2.com"
              ...
            }
          output: |
            http {
              url => "https://target3.com"
              ...
          }
    Note: Multiple offload targets that are defined in spec.external.offload2 are run in series. If offload to the first target in offload2 fails, then subsequent offloads are not run. Define your most important offload target first.
  5. If you want to offload API events or audit events selectively, use the offload filters as follows:
    Note: If API Connect is configured to send audit events to a registered analytics service and if offload filters are not used, the default offload includes both API events and audit events. When filter is used to offload a specific event, the offload includes only that event and all other events are discarded.
    1. To offload only API events, add a filter after the output plug-in configuration as in the following example:
      filter: |
        if "apicapievent" not in [tags] {
          drop {}
        } 
    2. To offload only audit events, add a filter after the output plug-in configuration as in the following example:
      filter: |
        if "audit" not in [tags] {
          drop {}
        } 
  6. Save the changes to your analytics CR. The ingestion pods restart automatically.
  7. Update your analytics subsystem infrastructure backup with your updated analytics CR. See Backup, restore, and disaster recovery.