Restarting Analytics pods and data collection

Restart the pods that support the internal communications within the Analytics subsystem in your API Connect deployment.

About this task

The internal certificates are used for all Analytics microservices to communicate with each other within the subsystem. The default names of the secrets for these are analytics-client, analytics-server, and analytics-ca.

When you refresh the internal certificates for the Analytics subsystem, restart the Analytics pods by completing the following steps.

Procedure

  1. If you did not already stop Analytics data collection, complete that task now.

    Data cannot be moved to storage while pods are unavailable and if you do not stop collection before restarting Analytics pods, that data is lost.

  2. Restart all of the Analytics pods by running the following command:
    kubectl -n namespace delete po -l app.kubernetes.io/instance=CR_name
    where:
    • namespace is the namespace where the Analytics subsystem is installed.
    • CR_name is the name of the CR for installing the Analytics subsystem (default name: analytics).
  3. Verify that all Analytics pods are ready by running the following command:
    kubectl -n namespace get po

    where namespace is the namespace where the Analytics subsystem is installed.

    Make sure that every pod is Running and Ready.

  4. If you disabled shard allocation, check the storage cluster's health by running the following command:
    Note: If you did not disable shard allocation, skip this step. If you stopped shard allocation, check the cluster's health before re-enabling shard allocation.
    kubectl -n namespace exec -it storage-master|shared_pod -- curl_es _cluster/health?pretty
    where:
    • namespace is the namespace where the Analytics subsystem is installed.
    • storage-master|shared_pod is the name of any storage-master or storage-shared pod. You can get the name of your storage pods by running the following command:
      kubectl get po -n namespace

    When you check the cluster's health, the response looks like the following example:

    {
      "cluster_name" : "apic-analytics-cluster",
      "status" : "yellow",
      "timed_out" : false,
      "number_of_nodes" : 1,
      "number_of_data_nodes" : 1,
      "active_primary_shards" : 318,
      "active_shards" : 318,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 630,
      "delayed_unassigned_shards" : 0,
      "number_of_pending_tasks" : 0,
      "number_of_in_flight_fetch" : 0,
      "task_max_waiting_in_queue_millis" : 0,
      "active_shards_percent_as_number" : 33.54430379746836
    }
    
    If you have a single node cluster, wait for the status to be yellow before proceeding to the next step. If you have a three node cluster, wait for the status to be green.
    Note: Single node analytics clusters never show green in the health output. Green indicates that the data is replicated to at least one other node, and since there are no other nodes in a single node cluster it remains as yellow.
  5. Associate the Analytics service with the Gateway service.
    Note: If you did not unassociate the Analytics service from the Gateway, skip this step.

    See Associating an analytics service with a gateway service.

  6. Enable shard allocation by running the following command:
    Note: If you did not disable shard allocation, skip this step.
    kubectl -n namespace exec -it storage-master|shared_pod -- curl_es _cluster/settings -XPUT -d '{"persistent":{"cluster.routing.allocation.enable":"all"}}'
    where:
    • namespace is the namespace where the Analytics subsystem is installed.
    • storage-master|shared_pod is the name of any storage-master or storage-shared pod. You can get the name of your storage pods by running the following command:
      kubectl get po -n namespace
    When you enable shard allocation, the response looks like the following example:
    {"acknowledged":true,"persistent":{"cluster":{"routing":{"allocation":{"enable":"all"}}}},"transient":{}}
  7. Check the storage cluster's health again and wait until the status is green.
  8. Verify that the Analytics service is working by completing the following steps:
    1. Invoke an API on your gateway that uses the associated Analytics subsystem.
    2. Use the Analytics user interface to confirm that new data was posted to the Analytics service.