401 self-signed certificate error returned when accessing Monitoring menus

A 401 self-signed certificate message is returned when you access Monitoring menus.

Problem

When you try to access Monitoring menus from the IBM Cloud Pak® for Multicloud Management console after IBM Cloud Pak® for Multicloud Management is deployed and the Monitoring operator (ibm-management-monitoring) is enabled, the following 401 self-signed certificate message is returned: IBM Cloud Event Management could not confirm your identity because the IBM Cloud Private management ingress certificate is not issued by an authorized authority. Your IBM Cloud Private administrator should review the use of the IBM Cloud Private Master Certificate Authority (masterCA) parameter for IBM Cloud Event Management."

Cause/Symptoms

This issue occurs when the IBM Cloud Pak® for Multicloud Management console certificate is invalid or suddenly changes. To verify this is the case, run the following commands.

oc get secret -n kube-public ibmcloud-cluster-ca-cert -o go-template='{{index .data "ca.crt"}}' | base64 -D > icp-ca.crt
curl --cacert icp-ca.crt https://CP_CONSOLE_HOST:443/idprovider/v1/auth/token

Note: Replace <CP_CONSOLE_HOST:443> with the real value of <CP_CONSOLE_HOST>. You can run the following command to determine what the value of <CP_CONSOLE_HOST> is:

  oc get route -n ibm-common-services cp-console

The result might be as follows:

  NAME         HOST/PORT                               PATH   SERVICES                 PORT    TERMINATION          WILDCARD
  cp-console   cp-console.apps.laamb.cp.fyre.ibm.com          icp-management-ingress   <all>   reencrypt/Redirect   None

If the previous command returns an SSL/TLS certificate problem type error, the following solution will not fix this issue. In this scenario, you probably used your own certificate and an error occurred during certificate configuration. To resolve, you can use a custom certificate. For more information, see IBM Certificate manager (cert-manager).

If some other errors are returned, complete the steps in the following solution to solve the issue.

Solution

  1. Check the cem-users pod logs to verify that a self-signed certificate error occurred:

    oc logs -l component=cem-users -n management-monitoring
    

    If output similar to: Error: SSL Error: SELF_SIGNED_CERT_IN_CHAIN exists in the output. You can confirm that a self-signed certificate error occurred.

  2. Restart the cem-users pod on the IBM Cloud Pak® for Multicloud Management hub cluster.

    oc delete pod -l component=cem-users -n management-monitoring
    
  3. Access the Monitoring menus from the IBM Cloud Pak® for Multicloud Management console again.