Unable to log in to web GUI on geo-redundant cluster

On attempting to log in to the web GUI on a geo-redundant cluster, the system goes into maintenance mode.

Problem

This happens when you log in to DASH on a geo-redundant cluster, and see a maintenance mode error as a result of another cluster having DASH artifacts created on it. This can also happen in reverse, where a backup cluster is down but restarted, while the primary cluster has DASH artifacts created on it. A sample error message is displayed:

[12/1/22 15:10:05:242 UTC] 00000145 ComponentServ W AbstractComponentServiceImpl getComponent Component named 'com.ibm.isclite.ISCAdminPortlet' could not be retrieved, even after reloading the registry.
[12/1/22 15:10:05:242 UTC] 00000145 ComponentServ W AbstractComponentServiceImpl getContextPath Component named 'com.ibm.isclite.ISCAdminPortlet' was not found, unable to retrieve a contextRoot value.
[12/1/22 15:10:05:242 UTC] 00000145 TreeFilter    E TreeFilter processDynamicFilter Exception:
                                 java.lang.NullPointerException
        at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:947)
        at com.ibm.isclite.service.datastore.topology.AbstractTopologyServiceImpl.getLayoutElement(AbstractTopologyServiceImpl.java:316)
        at com.ibm.isclite.service.datastore.topology.AbstractTopologyServiceImpl.getHiddenPage(AbstractTopologyServiceImpl.java:497)
        at com.ibm.isclite.common.util.Util.getPII(Util.java:199)
        at com.ibm.isclite.common.tree.TreeFilter.processDynamicFilter(TreeFilter

Resolution

The system goes into maintenance mode, and is a known issue. To force the system out of maintenance mode, first log in to the pod with the following command:

kubectl exec -ti {release name}-webgui-0 bash

The path for the WebGUI pod:

Cloud deployment: /home/netcool/app/jazz/ui/bin
Hybrid deployment: <JazzSM_home>/ui/bin for example /opt/IBM/JazzSM/ui/bin

For cloud deployments, run the following command from the /home/netcool/app/jazz/ui/bin directory. For hybrid deployments, run the following command from the <JazzSM_home>/ui/bin directory:

consolecli.sh ForceHAUpdate --username console_admin_user_ID --password console_admin_password

The ForceHAUpdate command pushes the local configuration to the database and updates the modules table to match the local node's module versions. Notifications are sent to other nodes to synchronize. Notified nodes with module versions that match the originating node are synchronized. Notified nodes with module versions that do not match, go into maintenance mode until an administrator updates their modules.