The opentt-ingester pod fails for OOMKilled after upgraded to Fix Pack 5

Sometimes you might find the opentt-ingester pod fails for OOMKilled after it's upgraded to Fix Pack 5 or Fix Pack 6. Follow the steps to solve this problem.

Symptoms

In the pod log, you can see the following information:

    Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    137
      Started:      Wed, 22 Jun 2022 08:53:47 -0400
      Finished:     Wed, 22 Jun 2022 10:08:14 -0400

Cause

The default memory limit is not enough for your environment.

Solution

You need to increase the memory limit to a bigger value. Follow the steps:

  1. Open the size0 configmap for editing by running the following command:

    oc edit cm monitoring-sizing-size0 -n management-monitoring
    
  2. Change the memory limit to a bigger value as you need (at least 250 MB). In the following setting, the memory limit is set to 250 MB.

    Deployment.opentt-ingester.ingester.memLimit: 250M
    
  3. Open the size1 configmap for editing by running the following command:

    oc edit cm monitoring-sizing-size1 -n management-monitoring
    
  4. Change the memory limit to a bigger value as you need (at least 500 MB). In the following setting, the memory limit is set to 500 MB.

    Deployment.opentt-ingester.ingester.memLimit: 500M