Managing resources for Cassandra and Inventory nodes

When you create a Pod, the Kubernetes scheduler selects a node for the Pod to run on. The scheduler ensures that, for each resource type, the sum of the resource requests of the scheduled containers is less than the capacity of the node. You can also colocate some services to be installed on the same node.

Symptoms

Typically, NFS container is installed on the node that has the highest resources in your cluster. In Telco Network Cloud Manager - Performance, Cassandra and Inventory containers are designed to co-locate with NFS node. If the NFS node does not have sufficient resources (CPU and memory), you might notice that Cassandra and Inventory Services might not start. In a cluster with six nodes, the following error might be seen: To
0/6 nodes are available: 1 node(s) didn't match node selector, 5 Insufficient memory.

Resolving the problem

To resolve this issue, follow the workaround:
  • Log in to Kubernetes Dashboard and change the namespace to npi.
  • Click Pods in Workloads pane.
  • Click the Actions icon (Actions icon) on Inventory Pod and select Edit.
  • Optional: Or, click the Inventory Pod, and then click Edit resource (Edit resource icon).

    The Edit a resource window opens.

  • Remove the following lines from the YAML file and click Update:
    nodeSelector:
        storage: nfs
  • If the Cassandra Service is still not started, then repeat step1 for Cassandra Service YAML file also.
  • Log in to your cloud web console and change the namespace to noi.

    https://<master_node_IP>:<Dashboard_externalPort>

  • Click Pods in Workloads pane.
  • Click the Actions icon (Actions icon) on Inventory Pod and select Edit.
  • Optional: Or, click the Inventory Pod, and then click Edit resource (Edit resource icon).

    The Edit a resource window opens.

  • Remove the following lines from the YAML file and click Update:
    nodeSelector:
        storage: nfs
  • If the Cassandra Service is still not started, then repeat step1 for Cassandra Service YAML file also.
CAUTION:
You might see a degradation in performance of both Inventory and Cassandra Services since the Pods must communicate across nodes.