Configuring load balancing for on-premises Web GUI or Dashboard Application Services Hub nodes (HA only)

Edit online

High availability icon Learn how to configure load balancing for the on-premises IBM® Netcool®/OMNIbus Web GUI or Dashboard Application Services Hub nodes in a hybrid high availability (HA) deployment, where there is more than one on-premises Web GUI node.

About this task

The on-premises Web GUI or DASH servers must be set up with load balancing by using an HTTP Server that balances the UI load. If you do not already have load balancing configured for your on-premises Web GUI or DASH nodes, then follow these steps.

Procedure

  1. Install Db2®.
    If you do not have a Db2 instance, install it on one of your on-premises servers. For more information, see https://www.ibm.com/docs/en/SSSHTQ_8.1.0/webtop/wip/task/web_con_downloaddb2.html
  2. Install IBM HTTP Server.
    If you do not have IBM HTTP Server then install it on one of your on-premises server. For more information, see https://www.ibm.com/docs/en/SSEQTJ_8.5.5/com.ibm.websphere.ihs.doc/ihs/welc6miginstallihsdist.html
  3. Configure WebSphere® certificates.
    1. Generate a Certificate Signing Request (CSR) on the second WebSphere node.

      On WebSphere node 2, use the WebSphere Administration console to navigate to Security - SSL certificate and key management - Key stores and certificates - NodeDefaultKeyStore - Personal certificate requests - New, and fill in the required fields, using the DASH domain name as the common name parameter value.

    2. Copy the generated certificate request file from WebSphere node 2 to WebSphere node 1.

    3. Generate a signed server certificate.

      Use the intermediate certificate on WebSphere node 1 to sign the CSR that was created in the previous step.

    4. Import the signed server certificate into WebSphere node 2's keystore.

      Copy the signed server certificate created in the previous step back to WebSphere node 2, and import it into WebSphere node 2's keystore, and then use the WebSphere Administration console to navigate to Security - SSL certificate and key management - Key stores and certificates - NodeDefaultKeyStore - Personal certificates - Recieve from certificate authority

    5. Add WebSphere node 1 intermediate CA certificate to WebSphere node 2's keystore.

      Copy the WebSphere node 1 intermediate certificate to WebSphere node 2, and then use WebSphere node 2's WebSphere Administration console to add this certificate to WebSphere node 2's keystore. Navigate to Security - SSL certificate and key management - Key stores and certificates - NodeDefaultKeyStore - Signer certificates -> Add and then add the intermediate CA certificate.

    6. Add WebSphere node 1 root CA certificate to WebSphere node 2's truststore.

      Copy the WebSphere node 1 root certificate to WebSphere node 2, and then use WebSphere node 2's WebSphere Administration console to add this certificate to WebSphere node 2''s keystore. Navigate to Security - SSL certificate and key management - Key stores and certificates - NodeDefaultTrustStore - Signer certificates -> Add and then add the root CA certificate.

    7. Update WebSphere node 2 to use the new certificate.

      Use WebSphere node 2'S WebSphere Administration console and navigate to SSL certificate and key management - Manage endpoint security configurations - JazzSMNode01. For inbound connections, set the Certificate alias in key store to the certificate that was added to the keystore in the previous step.

  4. Configure single sign-on for the DASH servers.
    Export the LPTA keys from the first DASH server, and then import them on to the other DASH server(s). For more information, see https://www.ibm.com/docs/en/SSSHTQ_8.1.0/webtop/wip/task/web_con_ssoprocedures.html.
  5. Create a database to manage load balancing, and then enable WebSphere to connect to the database.
    Create a database in Db2, and then from DASH, click Console Settings > WebSphere Administrative console > Launch WebSphere Administrative console, and then Resources > JDBC > JDBC providers and add an entry for Db2. For more information, see https://www.ibm.com/docs/en/SSEKCU_1.1.3.0/com.ibm.psc.doc/tip_original/ttip_config_ha_setup.html.
  6. Create a WebSphere datasource to enable connection to the load-balancing Db2 database:
    In DASH, click Console Settings > WebSphere Administrative console > WebSphere Administrative console, and then Resources > JDBC > Data Sources and add an entry for the load balancing Db2 database that you created.
  7. Create a key database for IBM HTTP Server to store keys and certificates in. 
    • cd /space/ibm/netcool/httpserver/bin
./gskcapicmd -keydb -create -db ~/http-server-keys -pw WebAS -stash
      For more information, see https://www.ibm.com/docs/en/SS7K4U_8.5.5/com.ibm.websphere.ihs.doc/ihs/tihs_createkeydb390.html
    • Add root CA cert to the IBM HTTP Server keystore.
      ./gskcmd -cert -add -db ~/http-server-keys.kdb -pw WebAS -file ~/root-ca.pem -label root-ca
    • Add intermediate cert to the IBM HTTP Server keystore.
      ./gskcmd -cert -add -db ~/http-server-keys.kdb -pw WebAS -file ~/intermediate-ca.pem -label intermediate
    • Create CSR
      ./gskcapicmd -certreq -create -db ~/http-server-keys.kdb -pw WebAS -dn "C=GB,ST=England,O=IBM,OU=HDM,CN=noi-on-prem1.fyre.ibm.com" -size 2048 -file ~/http-server-lb.csr -label http-server-lb
    • Sign the CSR with your intermediate cert to create http-server-lb.crt.
    • Add the signed cert to the IBM HTTP Server keystore.
      ./gskcmd -cert -receive -file ~/http-server-lb.crt -db ~/http-server-keys.kdb -pw WebAS
    • Assign the root CA certificate to be the default certificate.https://www.ibm.com/docs/en/SSEQTJ_8.5.5/com.ibm.websphere.ihs.doc/ihs/tihs_selfsigned.html Alternatively, the user can use the ikeyman utility provided with IBM HTTP server to assign the root CA certificate as the default
  8. Configure SSL for IBM HTTP server
    Locate the line # End of example SSL configuration in HTTP_server_install_dir/conf/httpd.conf, and then append the following, ensuring that your KeyFile and SSLStashfile values reference the key database file that you created for IBM HTTP Server.
    # End of example SSL configuration
LoadModule ibm_ssl_module modules/mod_ibm_ssl.so
Listen 443
<VirtualHost *:443>
SSLEnable
SSLProtocolDisable SSLv2
ErrorLog "/home/test/sslerror.log"
TransferLog "/home/test/sslaccess.log"
KeyFile "/home/test/http-server-keys.kdb"
SSLStashfile "/home/test/http-server-keys.sth"
</VirtualHost>
SSLDisable
  9. Tell IBM HTTP Server where the plugin-cfg.xml will be
    Add the following to the end of HTTP_server_install_dir/conf/httpd.conf
    LoadModule was_ap22_module "HTTP_server_install_dir/bin/64bits/mod_was_ap22_http.so"
WebSpherePluginConfig "HTTP_server_install_dir/config/plugin-cfg.xml"
  10. Configure the WebSphere Application Server plugin for IBM HTTP Server.
    Generate plugin-cfg.xml, and copy it to your IBM HTTP Server installation. 
    JazzSM_Profile/bin/GenPluginCfg.sh
cp /space/ibm/netcool/jazz/profile/config/cells/plugin-cfg.xml HTTP_web_server_install_dir/plugins/config/webserver1/plugin-cfg.xml
    Edit plugin-cfg.xml to point to your key store and stashfile (http-server-keys.kdb and http-server-keys.sth), and add entries for each of your DASH servers. For more information, see https://www.ibm.com/docs/en/SSEKCU_1.1.3.0/com.ibm.psc.doc/tip_original/ttip_config_loadbal_plugin_cfg.html
  11. Edit HTTP_web_server_install_dir/plugins/config/webserver1/plugin-cfg.xml
    Find the section called <UriGroup Name="server1_Cluster_URIs"> and append this line: 
    <Uri AffinityCookie="JSESSIONID_ibm_console_16310" AffinityURLIdentifier="jsessionid" Name="/oauth2/*"/>
  12. Start the HTTP Server 
    HTTP_web_server_install_dir/bin/apachectl start
  13. Stop and restart the Jazz® for Service Management application server 
    cd JazzSM_WAS_Profile/bin
./stopServer.sh server1 -username smadmin -password password
./startServer.sh server1
    where JazzSM_WAS_Profile is the location of the application server profile that is used for Jazz for Service Management. This is usually /opt/IBM/JazzSM/profile.
  14. Stop secondary DASH nodes and run the following command on the primary DASH node: 
    <JazzSM_Home>/ui/bin/consolecli.sh ForceHAUpdate --username <smadmin_user> --password <smadmin_password>
    Draft comment: DEIRDRELAWTON
    Nov 2020 Additional step #7918 and #7660

Results

When you have load balancing correctly configured, you are able to access DASH without providing a port in the URL, for example: https://http_server_hostname/ibm/console.
Verify that the DASH nodes are Active and InSync. Run the following command from one of the DASH servers:
_ui/bin/consolecli.sh ListHANodes --username --password
Where the admin user name is smadmin by default.
If the listed nodes are not Active and InSync, run the following command: 
consolecli.sh ForceHAUpdate --username --password
Check the load balanced setup. Check that the NODES table in the Db2 database associated with the jdbc/tipds data source, contains all HA nodes. Also check that the request sent to the load balancer returns all servers in the load balanced setup. Run the following command as an smadmin user: 
curl --insecure -X GET -v "https://<load-balancer-host>:<port>/ISCHA/NotificationServlet?username=smadmin&password=<smadmin-password>&listHANodes=truejsp"
Where values for your <load-balancer-host> and <port> are specified for your load balancer and <smadmin-password> is the password for the smadmin user.