Exporting Cloud Pak for Data audit records to LogDNA

You can export your Cloud Pak for Data audit records to LogDNA.

Before you begin

Best practice: You can run many of the commands in this task exactly as written if you set up environment variables for your installation. For instructions, see Setting up installation environment variables.

Ensure that you source the environment variables before you run the commands in this task.

About this task

Cloud Pak for Data uses the LogDNA Fluentd plugin to integrate with LogDNA.

To export your audit records to LogDNA, complete the following tasks:

  1. Get the Fluentd configuration snippet to connect to your LogDNA organization.
  2. Configure Cloud Pak for Data to send events to LogDNA.

    Choose the appropriate option for your environment:

Get the Fluentd configuration snippet to connect to your LogDNA organization

In the LogDNA client:

  1. On the web client home page, select Fluentd in the Via Platform section.
  2. Copy or download configuration snippet from the If you use Fluentd section.

    The snippet has the following format:

    <match your_match>
@type logdna
api_key YOUR-APIKEY
hostname "#{Socket.gethostname}"    # your hostname (required)
app my_app                          # replace with your app name
#mac C0:FF:EE:C0:FF:EE              # optional mac address
#ip 127.0.0.1                       # optional ip address
</match>
  3. Edit the snippet:
    1. Replace the opening and closing <match> tags with <store> tags.
    2. Add the following parameters after the ip entry:
      buffer_chunk_limit 1m                # optional: Parameter to improve performance
flush_at_shutdown true               # optional: Parameter to improve performance
    3. Remove the #mac and #ip entries from the snippet

    The updated snippet should have the resemble the following example:

    <store>
@type logdna
api_key YOUR-APIKEY
hostname "#{Socket.gethostname}"    # your hostname (required)
app my_app                          # replace with your app name
buffer_chunk_limit 1m               # optional: Parameter to improve performance
flush_at_shutdown true              # optional: Parameter to improve performance
</store>
  4. Save the updated snippet.

Edit the zen-audit-config configmap

You can edit the existing zen-audit-config configmap to connect to your LogDNA environment.

From the Cloud Pak for Data cluster:

  1. Change to the project where Cloud Pak for Data is installed:
    oc project ${PROJECT_CPD_INSTANCE}
  2. Make a backup of the current zen-audit-config configmap.
  3. Edit the zen-audit-config config map:
    oc edit configmap zen-audit-config
  4. Update the configmap:
    1. Add the <store> configuration to the configmap.
      Tip: The zen-audit-config configmap includes a sample LogDNA configuration, which is commented out by default.

      The <store> configuration must be inside the <match export export.** records records.** syslog syslog.**> tag and after the @type copy tag.

      Ensure that the entry is indented correctly.

    2. Replace the following variables with the appropriate values for your environment:
      <CPD-HOSTNAME>
      Replace <CPD-HOSTNAME> with the hostname of the Cloud Pak for Data server.

      This value appears in the records that are sent to LogDNA. This information helps you determine which Cloud Pak for Data instance the audit records are coming from.

      <ZEN-AUDIT>
      Replace <ZEN-AUDIT> with the name that you use for Cloud Pak for Data.

      This value appears in the records that are sent to LogDNA. This information helps you understand which application the audit records are coming from.

    3. If you want to use TLS to connect to LogDNA, replace <XXXXXXXXX> with the contents of the LogDNA TLS certificate.

    The contents of the configmap depend on whether you want to use TLS:

    Configmap for connecting to LogDNA using TLS (recommended) 
    apiVersion: v1
metadata:
  name: zen-audit-config
data:
  ca.pem: |
    -----BEGIN CERTIFICATE-----
     <XXXXXXXXX>
    -----END CERTIFICATE-----
  fluent.conf: |-
    <match export export.** records records.** syslog syslog.**> 
      @type copy
      <store>
        @type logdna
        api_key YOUR-APIKEY                # API key from snippet
        ingester_domain HOST               # LogDNA ingester domain address
        hostname <CPD-HOSTNAME>
        app <ZEN-AUDIT>
        buffer_chunk_limit 1m
        flush_at_shutdown true
        tls true                           # Required to use TLS
        ca_file /fluentd/config/ca.pem     # Required to use TLS; specify the cert in the ca.pem section
      </store>
    </match>
    Configmap for connecting to LogDNA without TLS 
    apiVersion: v1
metadata:
  name: zen-audit-config
data:
  fluent.conf: |-
    <match export export.** records records.** syslog syslog.**> 
      @type copy
      <store>
        @type logdna
        api_key YOUR-APIKEY                # API key from snippet
        ingester_domain HOST               # LogDNA ingester domain address
        hostname <CPD-HOSTNAME>            
        app <ZEN-AUDIT>
        buffer_chunk_limit 1m
        flush_at_shutdown true
      </store>
    </match>
  5. Save the changes to the zen-audit-config configmap. For example, if you are using vi, hit esc and enter :wq.
  6. Delete all zen-audit pods to force a restart to pick up changes:
    1. Get the list of zen-audit pods:
      oc get pods | grep zen-audit
    2. Delete the zen-audit pods.
      Tip: If there are multiple zen-audit pods, delete each pod individually to avoid a forced restart.
      oc delete pods zen-audit-xxxxxx-xxx

      Replace zen-audit-xxxxxx-xxx with the ID returned by the oc get pods command.

Create a custom configmap

You can create a custom configmap to connect to your LogDNA environment.

From the Cloud Pak for Data cluster:

  1. Change to the project where Cloud Pak for Data is installed:
    oc project ${PROJECT_CPD_INSTANCE}
  2. Set the following environment variables:
    1. Set CONFIGMAP_NAME to the name that you want to use for the custom configmap:
      export CONFIGMAP_NAME=<configmap-name>
    2. If you want to use TLS, set TLS_CERT to the TLS certificate:
      export TLS_CERT=<certificate>
    3. Set CPD-HOSTNAME to the hostname of the Cloud Pak for Data server:
      export CPD-HOSTNAME=<hostname>

      This value appears in the records that are sent to LogDNA. This information helps you determine which Cloud Pak for Data instance the audit records are coming from.

    4. Set ZEN-AUDIT to the name that you use for Cloud Pak for Data.
      export ZEN-AUDIT=<app-name>

      This value appears in the records that are sent to LogDNA. This information helps you understand which application the audit records are coming from.

  3. Create the configmap:
    Configmap for connecting to LogDNA using TLS (recommended) 
    cat <<EOF | oc apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: ${CONFIGMAP_NAME}
  namespace: ${PROJECT_CPD_INSTANCE}
data:
  ca.pem: |
    -----BEGIN CERTIFICATE-----
     ${TLS_CERT}
    -----END CERTIFICATE-----
  fluent.conf: |-
    <match export export.** records records.** syslog syslog.**> 
      @type copy
      <store>
        @type logdna
        api_key YOUR-APIKEY                # API key from snippet
        hostname ${CPD-HOSTNAME}
        app ${ZEN-AUDIT}
        # mac C0:FF:EE:C0:FF:EE            # optional mac address
        buffer_chunk_limit 1m
        flush_at_shutdown true
        tls true                           # Required to use TLS
        ca_file /fluentd/config/ca.pem     # Required to use TLS; specify the cert in the ca.pem section
      </store>
    </match>
EOF
    Configmap for connecting to LogDNA without TLS 
    cat <<EOF | oc apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: ${CONFIGMAP_NAME}
  namespace: ${PROJECT_CPD_INSTANCE}
data:
  fluent.conf: |-
    <match export export.** records records.** syslog syslog.**> 
      @type copy
      <store>
        @type logdna
        api_key YOUR-APIKEY                # API key from snippet
        hostname ${CPD-HOSTNAME}            
        app ${ZEN-AUDIT}                
        # mac C0:FF:EE:C0:FF:EE            # optional mac address
        buffer_chunk_limit 1m
        flush_at_shutdown true
      </store>
    </match>
EOF
  4. Patch the zenService custom resource to specify the custom configmap:
    oc patch zenService lite-cr \
-n ${PROJECT_CPD_INSTANCE} \
--type='json' \
--patch '[{"op": "replace", "path":"/spec/zen_audit_config_custom", "value": '${CONFIGMAP_NAME}' }]'
  5. Delete all zen-audit pods to force a restart to pick up changes:
    1. Get the list of zen-audit pods:
      oc get pods | grep zen-audit
    2. Delete the zen-audit pods.
      Tip: If there are multiple zen-audit pods, delete each pod individually to avoid a forced restart.
      oc delete pods zen-audit-xxxxxx-xxx

      Replace zen-audit-xxxxxx-xxx with the ID returned by the oc get pods command.

Results

After the changes are applied, new audit events from Cloud Pak for Data are sent to LogDNA.

For more information about the audit events, see: