Empty Analytics display on OpenShift and Cloud Pak for Integration
If your API Connect analytics data doesn't display in the Analytics dashboard, complete the following steps to determine the cause of the problem.
Procedure
-
Run the following command to check the logs of the mtls-gw pod:
oc logs analyt-mtls-gw-7665f5d445-4cx4n -n <APIC_namespace>
The response looks like the following example:
10.254.6.1 - - [22/Sep/2021:18:05:54 +0000] "POST /x2020/v1/apievent/_bulk HTTP/1.1" 200 2 "-""-" 2841 0.072 0.070
-
Check the response for any
_bulk
request coming from the gateway, as shown in the example.If there are no
_bulk
requests, then the problem is with the gateway and the wsm-agent, which is not sending bulk requests to the Analytics server. -
If you see the
_bulk
request., then check the return code in the response.If
_bulk
requests are processed successfully you will see a return code 200 (as in the example), which indicates that the bulk request has been ingested. -
If it is any other return code, check the ingestion pod logs for any errors, which will
indicate any issues during ingestion:
oc logs analyt-ingestion-78d7cff7f7-6pktd
-
Check the response for any
-
If everything with the data flow looks fine, and if you are unable to see analytics data, then
check the health of Analytics by executing into one of the storage pods:
oc exec -it <storage_pod> — bash d.bash-4.4$ curl_es _cluster/health?pretty=1
For example:
{ "cluster_name" : "apic-analytics-cluster", "status": "yellow", "timed_out": false, "number_of_nodes": 3, "number_of_data_nodes": 1, "active_primary_shards": 258, "active_shards": 258, "relocating_shards": 0, "initializing_shards": 0, "unassigned_shards": 510, "delayed_unassigned_shards": 0, "number_of_pending_tasks": 0, "number_of_in_flight_fetch": 0, "task_max_waiting_in_queue_millis": 0, "active_shards_percent_as_number": 33.59375 }
-
Check the status of the cluster (red, yellow, green) in the response.
In the example, the status is set to yellow.
- Green indicates all indices are fine.
- Yellow indicates there are some unassigned shards, which might still be replicating.
- Red indicates there is a problem with one or more of the indices.
-
Run the following command to get a list of indices:
bash-4.4$ curl_es _cat/indices?v
- In the list, find the red indices.
-
Delete each red index by running the following command:
bash-4.4$ curl_es -XDELETE <index_name>
-
Check the status of the cluster (red, yellow, green) in the response.