Technical Blog Post
Healthcheck for DASH High Availability Setup
There are many time or scenario that you need to check the status of your DASH HA clusters after you had perform fix pack upgrade or stop the server for maintenance.
In the normal situation, these results will be shown if the clusters are working properly.
Run the JazzSM/profile/ui/bin/consolecli.sh ListHANodes --username smadmin --password smadmin
If the ListHANodes are showing Java null error, please do ensure that your Db2 server is reachable and is started properly via the db2start command
If the LIstHANodes are returning ACTIVE and InSync but pages or roles are not sync properly between servers, then please check the Systemout.log for any SSL Handshake errors.
Please refer to this technote to fix the SSL certificated issue.
If your ListHANodes return error message,
For Unreachable message, make sure that your JazzSM server is started properly via the startServer.sh server1 command and that you can login to the DASH URL
link via https://<yourhostname>:16311/ibm/console
Is the LIstHANodes are returning INACTIVE and NodeSync are showing Unreachable and the DASH URL can be accessed, please check if your server name have FQDN.
It cannot be only server1.test and should be server1.test.ibm.com in the host file name and JazzSM_HOME/profile/config/cells/JazzSMNode01Cell/Nodes/JazzSMNode01/serverindex.xml file
If the NodeSync is showing INACTIVE,
You can perform these command to refresh the node with the latest content from database.
The ForceHARefresh exports data from the database and imports it locally. The database module version for Dashboard Application Services Hub must be lower than local node for export and import to succeed. Use this command to upgrade a cluster when there are no nodes that are in synch with the database module versions. Use the ForceHARefresh command, verify the result, and then use the ForceHAUpdate command to update the database.
./consolecli.sh ForceHARefresh --username smadmin --password netcool
To force a database update after running the ForceHARefresh command, as an administrator, run the ForceHAUpdate command. The ForceHAUpdate command pushes the local configuration to the database and updates the modules table to match the local node's module versions. Notifications are sent to other nodes to synchronize. Notified nodes with module versions that match those of the originating node are synchronized. Notified nodes with module versions that do not match, go into maintenance mode until an administrator updates their modules accordingly.
./consolecli.sh ForceHAUpdate --username smadmin --password netcool
If you encounter these problem where you can't create a page on the cluster, you would need to run this command
JazzSM/profile/ui/bin/consolecli.sh Unlock --username smadmin --password netcool
Additionally, you can also check if these tables are created and updated properly in the DB2. Please use the db2inst1 user that is used to create this Db2 database.
Enter Db2 on the Unix command line and then connect to the database used for DASH. In the following we assume that database name is DASH:
db2inst1:> db2 db2 => connect to DASHDB Run below command to list tables in the database. You should see below tables. db2 => list tables GLOBAL_CONFIG MODULES NODES NODES_STORES STORES
Below SQL can be used to list the number of nodes in the cluster.
db2 => SELECT * FROM NODES
It should return the numbers of nodes cluster with the right IP hostname that you had configured the clusters servers.