Configuring offload on Kubernetes and OpenShift

Configure your analytics subsystem to offload API event data to a third-party system.

Procedure

  1. Edit your analytics CR:
    kubectl edit a7s
    Note: If you are using the OpenShift top-level CR deployment, edit your APIConnectCluster CR instead. Updates should be made in the spec.analytics section.
  2. Set spec.external.offload to true, and add the offload plugin configuration:
    The following example shows an HTTP output plugin:
    spec:
      external:
        offload:
          enabled: true
          output: |
            http {
              url => "https://example.com"
              http_method => "post"
              ssl_verification_mode => "none"
              id => "API Connect analytics event data"
            }

    In this example, the URL points to an HTTPS endpoint, but verification of the endpoint's server certificate is disabled by setting ssl_verification_mode to "none". If you want your analytics subsystem to verify the endpoint server certificate, then see Configuring offload server certificates on Kubernetes and OpenShift.

    The id property is not mandatory, but it is recommended.

    For more information about the HTTP output plugin configuration, see the Logstash HTTP plugin documentation.

  3. Optional: Add a filter after the output plugin configuration to modify data before it is sent to the offload target.
    The following example filter removes the query_string field:
    spec:
      external:
        offload:
          enabled: true
          output: |
            http {
              ...
            }
          filter: |
            ruby {
              remove_field => ["query_string"]
            }
    For information about modifying offloaded data, see Customizing data with ingestion filters.
  4. Optional: To configure a second offload target, create a new spec.external.offload2 section and add the configuration for this second target to this section.
    If you have three or more offload targets, then you can put multiple configurations in spec.external.offload2. For example:
    spec:
      external:
       offload:
          enabled: true
          output: |
            http {
              url => "https://target1.com"
              ...
            }
        offload2:
          enabled: true
          output: |
            http {
              url => "https://target2.com"
              ...
            }
          output: |
            http {
              url => "https://target3.com"
              ...
          }
    Note: Multiple offload targets defined in spec.external.offload2 are run in series. If offload to the first target in offload2 fails, then subsequent offloads are not run. Define your most important offload target first.
  5. Save the changes to your analytics CR. The ingestion pods restart automatically.
  6. Update your analytics subsystem infrastructure backup with your updated analytics CR. See Backup, restore, and disaster recovery.