MongoDB pod hangs after installation

One of the MongoDB pods hangs after the installation of foundational services 3.8.x.

Symptom

When you install foundational services on OpenShift 4.6 or 4.7 on Linux on Power (ppc64le) or IBM Z architecture, one of the MongoDB pods hangs, and remains in the Init:1/2 status. The following information shows in logs:

$oc logs icp-mongodb-2 -c bootstrap
2021/06/22 18:24:47 Determined Domain to be ibm-common-services.svc.cluster.local
2021/06/22 18:24:57 lookup icp-mongodb on 192.0.2.*: no such host
2021/06/22 18:25:09 lookup icp-mongodb on 192.0.2.*: no such host
2021/06/22 18:25:20 lookup icp-mongodb on 192.0.2.*: no such host

Cause

This is a known issue related to the DNS setting.

Resolving the problem

Complete the following steps to resolve the issue:

1. Use the following command to scale down the dns-operator.

    oc -n openshift-dns-operator scale deployment/dns-operator --replicas=0
   

2. Run the following command to edit the dns-default confi gmap. Remove the bufsize 1232 entry from the map.

    oc edit cm -n openshift-dns dns-default
   

3. Restart the DNS pods. You might restart all the DNS pods, or just the one for the node where the mongoDB pod is stuck.

    oc delete pod -n openshift-dns --all