Scaling log anomaly detection pods

You can scale up the number of log anomaly detection pods for your instace.

Each log-anomaly-detection replica supports roughly 10 components. By default, a starter deployment has 2 log-anomaly-detection pods, while a production deployment has 4.

To scale the replicas to handle processing for additional components, modify the AIManagerMainProd custom resource.

Before you begin

Clusters can be for starter or production deployments, as described in Hardware requirements. A starter deployment cluster has limited resources, so you might want to check your cluster resources before you complete the following task to ensure that you are able to allocate adequate resources.

Procedure

  1. Log in to your cluster by running the oc login command.

    oc login -u kubeadmin -p <password>
    

    For more information, see Logging in to the OpenShift CLI.

  2. Edit the IBM Cloud Pak for AIOps customer resource. The following command opens the file in the vi editor for editing.

    oc edit AIManagerMainProd aimanager
    
  3. Search for the following configuration:

    spec:
      helmValues:
        global:
          logAnomaly:
            replicas: 2
    
  4. Modify the replica count, such as to change the number of replicas to 4.

    spec:
      helmValues:
        global:
          logAnomaly:
            replicas: 4
    

    Note: Each instance of the log-anomaly-detector pod requires an additional 0.5 CPU cores and 0.5Gi of memory.

  5. Save the file.

  6. Restart the ai-platform-api-server pod to put the changes into effect. Restart the pod by deleting the associated pod. A new pod automatically starts up and uses the new parameters that are specified in the configmap.

    1. First, determine the name of the currently running pod.

      oc get po | grep ai-platform-api-server
      

      Your output can resemble the following sample output:

      aimanager-aio-ai-platform-api-server-7d75768cbc-zx5dh             1/1     Running     0          6d1h
      

      Where,

      • The string aimanager-aio-ai-platform-api-server-7d75768cbc-zx5dh is the pod ID. The character string 7d75768cbc-zx5dh uniquely identifies the pod. This string is the value for <unique pod reference> in the following steps.
      • The other information indicates that the pod is running and has been running for 6 days and 1 hour.
    2. Delete the pod by running the following command:

      oc delete po aimanager-aio-ai-platform-api-server-7d75768cbc-zx5dh
      
    3. Wait a short time, then run the following command to check that the new pod started up, and to show the new pod ID.

      oc get po | grep ai-platform-api-server