Common issues and solutions

Symptom: The Event Manager instance pod logs show the following error:

Certificate Manager is not present. This needs to be installed and the operator pod restarted for reconciles to continue.

Cause: The Event Endpoint Management operator was installed before the 'cert-manager', and the operator is unable to detect the now installed certificate manager.

Solution:

1. Install a certificate manager. For more information about installing the cert-manager Operator for Red Hat OpenShift, see Installing the operators by using the Red Hat OpenShift console or Installing the operators by using the CLI.

2. Restart the eem-operator pod so the reconcile can continue.

Symptom: After a period of inactivity the browser can timeout on the "Select an OpenAPI Specification" page, displaying the following error: could not add asset, parent window missing

Solution: Reload the "Select an OpenAPI Specification" page.

Symptom: When running a docker run command, you get the following error:

docker run "/kube/config": open /kube/config: permission denied

Cause: Read-write permissions are needed for KUBECONFIG (~/.kube/config).

Solution: Run the following to give the user read-write permissions to the file:

chmod +rw ~/.kube/config

Symptom: When following the instructions for "Generating an upgrade plan using the CLI" in Upgrading from version 16.1.0 or 16.1.3 for an online (connected) installation, you are unable to generate the upgrade plan.

Cause: You may not have the correct configuration or permissions for Docker or Podman, or there is an error in the KUBECONFIG command.

Solution:

• On your online (connected) OpenShift cluster, run the oc admin command:

  oc adm must-gather --image=icr.io/cpopen/ibm-integration-upgrade-must-gather:v4 -- /usr/bin/gather --namespace openshift-operators --to 16.2.0 --verbose

Issue: Clicking Monitoring in the Platform UI returns "Entity not found".

Cause: To diagnose the issue, confirm there's a connection between the instance and the Instana agent.

1. Sign in to the Instana user interface.

2. Click Infrastructure > Entity Explore > Search/Find Capability > view instance. If you are unable to find the instance, there is an issue with the connection between the Instana agent (sensor) and the instance.

3. Identify the nodes that the instance is running on, then find the instana-agent pods running on the same node to get the sensor-related logs for the instance.

Solution: Review the Instana configuration for the instance and refer to the applicable configuration guide for the monitored instance.

Symptom: OpenShift creates FailedToRetrieveImagePullSecret warning events for Cloud Pak for Integration pods, even though pods are running successfully. For example:

Events:
Type     Reason                           Age                    From     Message
----     ------                           ----                   ----     -------
Warning  FailedToRetrieveImagePullSecret  89s (x15803 over 13d)  kubelet  Unable to retrieve some image pull secrets (ibm-entitlement-key); attempting to pull the image may not succeed.

Cause: A change in OpenShift version 4.15 and later produces warning events when a pull secret is missing, even if the image pull is successful and the pod is running correctly. Previously, missing pull secrets were silently ignored. Cloud Pak for Integration automatically adds pull secret references with known names to pods to make it simple to add pull secrets for use with Cloud Pak for Integration. If you are not using these pull secrets, they now trigger the warning events.

If you are using a global pull secret, the ibm-entitlement-key secret may not exist in the same namespace as the pod, triggering the warning. You might use a global pull secret for these reasons:

• To provide authentication to a local registry mirror for a network-restricted environment.

• To simplify pull secret management for an online cluster. Other pull secrets added to Cloud Pak for Integration pods by default may also trigger the warning.

Solution: If the image has been pulled successfully, you can ignore the warnings. You can also suppress the warnings by creating an empty secret. Get the list of missing pull secrets from the warning message and create them by running the following command for each missing pull secret:

oc create secret generic [secret-name] -n [namespace]

For example:

oc create secret generic ibm-entitlement-key -n integration

Symptom: The Keycloak banner does not display correctly.

Cause: Because of an issue with the display name in the HTML, the banner isn't replaced by the CSS.

Solution: Follow these steps:

1. Log in to the Keycloak admin interface (from the Platform UI, click the navigation menu icon, then click Administration > Access control).

2. In the navigation pane, click to expand the dropdown menu and select the IBM Cloud Pak theme.

3. Click Realm settings.

4. Under the general tab, set the HTML Display name to <div class="kc-logo-text"><span>IBM Cloud Pak</span></div> and click Save.

Symptom: Operators are in an unknown state, a log error states that a dependency is not satisfied, and the catalog-operator-* pod in the openshift-operator-lifecycle-manager namespace has multiple restarts.

Cause: The Operator Lifecycle Manager (OLM) in OpenShift is limited to processing a single operator request at a time.

Solution: To resolve, delete all pods in the openshift-operator-lifecycle-manager namespace, which triggers a process to recreate the pods and OLM to progress the installation of the operators:

oc delete pods --all -n openshift-operator-lifecycle-manager

Symptom: During an OADP restore, the Cp4iServicesBinding resource generates an error, mentioning an integration-keycloak-user-secret secret that is missing. This issue occurs with version 7.3.15 or earlier of the IBM Cloud Pak for Integration operator.

Cause: The "internal-keycloak-user-secret" secret does not get backed up or restored.

Solution: Complete the following steps:

1. In the Keycloak Admin Console, confirm that the master realm is the current realm, and delete the user with the name internal-keycloak-user.

   Tip: If master is not the current realm, click Manage realms in the navigation panel, then click master to change.

   1. Log in to the Keycloak Admin Console. For instructions, see "Advanced Keycloak administration" in Keycloak configuration. Skip the last step to change the realm setting, since the correct setting here is master.

   2. In the navigation panel, click Users.

   3. In the row for internal-keycloak-user`, click the overflow memu (three-dot) icon and select Delete.

2. In the OpenShift web console, delete the internal-keycloak-user Kubernetes resource of type IntegrationKeycloakUser.

   1. Log in to the OpenShift web console with adminsistrator credentials.

   2. In the navigation panel, click Home > Search.

   3. In the Resources search box, enter IntegrationKeycloakUser and select internal-keycloak-user.

   4. In the row for internal-keycloak-user, click the overflow (three-dot) menu icon and select Delete.

Symptom: When you log in to the Platform UI as an integration-admin user, you get a page not working message and you are redirected to a 401 unauthorized page.

Cause: You have assigned all available roles to the integration-admin user, which makes the token or cookie too large.

Solution: Un-assign any roles that were manually added to this user. The default roles allow this user to perform all actions, so no additional roles are needed.

Symptom: You diagnose the pod status by entering the following command, where the specified value for pod is ibm-integration-platform-navigator-operator with the pod name (user ID) appended. For example, you could enter ibm-integration-platform-navigator-operator-8d99d9d987-9q2fm.

oc describe pod <ibm-integration-platform-navigator-operator-podname> -n <namespace>

In the response, you get OOMKilled in the description of the pod.

Cause: The ibm-integration-platform-navigator-operator container in the Platform UI operator pod is exceeding its allocated memory limit and therefore gets terminated.

Solution: Increase the memory limits of the Platform UI operator pod by updating the subscription.spec section:

config:
   resources:
      requests:
         memory: "256Mi" 
      limits:
         memory: "1Gi" 

Do not change the requests.memory value. Increase the provided value (1Gi) for limits.memory until the pod is running.