No analytics data in UIs

How to check that your analytics subsystem is receiving API event data from the gateways.

Note: VMware users: The operations to confirm API event data is arriving from the gateways must be run from inside the analytics VM. To access your analytics VM:
  1. Use an SSH client to login as the apicadm user: 
    ssh apicadm@<analytics VM FQDN>
  2. sudo to root:
    sudo -i

The kubectl commands in these operations specify a <namespace> argument. On VMware, replace <namespace> with default. Alternatively you can omit -n <namespace> from the commands completely.

  1. The mtls-gw is the pod that receives the API events from the gateway. Identify the name of your mtls-gw pod or pods with:
    kubectl get pods -n <namespace>
    Expected output:
    NAME                                                              READY   STATUS      RESTARTS   AGE
...
analyt-mtls-gw-7665f5d445-4cx4n                                      1/1     Running     0          3d19h
  2. For each mtls-gw pod, run the following command to see the logs of that pod:
    kubectl logs <mtls-gw pod name> -n <namespace>
  3. The gateway sends API event data to the mtls-gw pod as an HTTP POST call to the path /x2020/v1/apievent/_bulk.
    Every time that the mtls-gw pod receives API event data from the gateway, it logs the following message, known as a _bulk message: 
    10.254.6.1 - - [22/Sep/2021:18:05:54 +0000] "POST /x2020/v1/apievent/_bulk HTTP/1.1" 200 2 "-""-" 2841 0.072 0.070

    Check the mtls-gw pod logs for recent _bulk messages.

    If you find no _bulk messages in the mtls-gw logs, then the analytics subsystem is not receiving API event data from the gateway. Check the network connection between your gateway and analytics subsystems, and review Analytics endpoints in the gateway documentation.

    The HTTP return code is shown in the mtls-gw log messages. In the example, this code is 200, which means the API event data was received successfully by the analytics subsystem. Whatever the return code is, if your mtls-gw pod logs contain recent _bulk messages and you do not see any analytics data for this time period in the UI, then gather postmortem logs and open an IBM support case.