Cluster monitors

A cluster monitor is clustering technology used by Db2® Mirror to determine node status from the Hardware Management Console (HMC). Cluster monitors are defined during the Db2 Mirror configuration process. A cluster monitor uses the REST server on the HMC attached to the node to be monitored. Once defined, the HMC can be queried for the status of the attached node by Db2 Mirror when needed.

The preceding figure shows a cluster monitor configuration where Node 1 is monitoring Node 2 and vice versa. The figure shows that each node is attached to different HMCs, but both nodes could be attached to the same HMC. If Node 1 loses communication with Node 2, it will query the HMC through the cluster monitor to get the status of Node 2. If the HMC reports that Node 2 is down, then node failure processing will occur. If the HMC reports that Node 2 is available, then Db2 Mirror assumes there is a problem with communication between the nodes and will treat this as a network outage.

The cluster monitor is also used by registered IFS IASPs to determine when to mutate from a client file system instance to a server file system or when to mutate from a server file system instance to a stand-alone file system.

A cluster monitor is required on each node in the Db2 Mirror pair. The cluster monitor contains the host name of the HMC that is attached to the other node in the pair, and a user and password to access the HMC.

If a third node has been added to the cluster for use as a quorum node, you can also define cluster monitors on that third node to monitor both of the Db2 Mirror nodes. The next figure shows this configuration.

Figure 2. Quorum node monitoring Db2 Mirror pair

Monitoring the Db2 Mirror node status from the quorum node allows Db2 Mirror to determine the correct state of the nodes in a situation where both nodes become unavailable at the same time.

Without monitoring from the third node, both nodes would have a state of BLOCKED when they become available again and the administrator would need to determine the correct primary node prior to resuming replication.
With cluster monitors on the third node, Db2 Mirror can distinguish between whether both nodes became unavailable at the same time or if there was a communication problem where one node might have continued data changes. With this knowledge, the nodes can safely resume their previous roles.

Setting up the cluster monitors is done as part of Db2 Mirror configuration. If cluster monitors were not originally configured and must be added or changed, the Digital Certificate Manager (DCM) *SYSTEM password is required to store and retrieve the digital certificate from the HMCs. For more information, see Digital certificate for cluster monitors.

Managing cluster monitors using the Db2 Mirror GUI

To use the Db2 Mirror GUI to show the cluster monitors which are currently configured, select Manage Cluster from the Setup icon on the left side of the GUI.

Figure 3. Manage cluster from Db2 Mirror GUI

By clicking on a node in the list, you will be shown the cluster monitors currently configured on that node. You can add or delete a cluster monitor. If the HMC has changed, or if the user or password has changed, you must delete the current cluster monitor and add it again with the correct HMC name, user, and password.

Figure 4. Cluster management window in Db2 Mirror GUI

Managing cluster monitors using the db2mtool command

To show what is currently configured for cluster monitors, you can use the db2mtool Qshell command with the action=cluster and cl=lstclumon parameters.

db2mtool action=cluster cl=lstclumon node=<node>

This will display the monitoring node, the HMC host name being queried, and the HMC user as shown here (with some information obfuscated).

Figure 5. Current cluster configuration using db2mtool

To add a new cluster monitor, use the db2mtool Qshell command with the action=cluster and cl=addclumon parameters. For the node parameter, specify the node that is doing the monitoring. The restxxx parameters are the HMC hostname, user, and password for the HMC attached to the node being monitored.

For example:

db2mtool action=cluster cl=addclumon node=NodeA  
    restsvr=HMC2 restuser=xxx restpassword=xxx

To change a cluster monitor, such as when a node is attached to a different HMC or if the user or password changes on the HMC, use the db2mtool Qshell command with the action=cluster and cl=chgclumon parameters. For example:

db2mtool action=cluster cl=chgclumon node=NodeA restsvr=HMC2 
    newrestsvr=HMC3 newrestuser=xxx newrestpassword=xxx

Managing cluster monitors using PowerHA

If the IBM® PowerHA® SystemMirror® for i product (5770-HAS) is installed, there are PowerHA commands to add (ADDCLUMON) and change (CHGCLUMON) cluster monitors. The Work with Cluster (WRKCLU) command can be used to list the cluster monitors defined on a node.