Installing, Configuring, and Testing IBM Tivoli Network Manager IP Edition and Tivoli Business Service Manager Failover
This document includes procedures for installing and configuring a failover environment where IBM® Tivoli® Network Manager IP Edition (Network Manager) and Tivoli Business Service Manager (TBSM) have a common object server (OMNIBus) and Tivoli Integrated Portal server.
The following figure shows the primary and secondary server tiers.
Figure 1: TBSM and ITNM failover environment
Failover includes:
- Object server failover
- Data server failover for Network Manager IP
- Data server failover for TBSM
- High availability for Tivoli Integrated Portal
In this document you use the Network Manager IP set up utility to install and configure the common object server and Tivoli Integrated Portal server. The same Network Manager IP and TBSM installers are used to install the data servers.
Important: For integration, TBSM Version 4.2 and Network Manager IP Version 3.8 were tested.
Installation and Failover Configuration Procedure
Follow these steps to install and configure a distributed TBSM and Network Manager IP environment with failover.
Installing and configuring failover for the Secondary Object Server
- On the sun25-z00 server (see Figure 1) and as the non-root user, start the Network Manager IP installer.
- Select Custom Installation and click Next.
- Select Multi-server installation and *Customize settings and click Next.
*In this example, FIPS Compliance is not selected. You can enable FIPS, but you will not be able to use MySQL because it does not support FIPS.
- Select the object server by selecting only Install event management software and click Next.
- Now, complete the installation wizard. The following settings were used in integration testing:
- Run <install_dir>/netcool/omnibus/bin/nco_xigen.
- Remove any entries in the server list for NCOMS or NCO_GATE. Other entries in the server list, such as the NCO_PA and NCO_PROXY entries placed in the table by the installer, are not necessary for TBSM failover. They can, however, be used by other functions or products using the ObjectServer server.
- Specify information about the primary ObjectServer server:
- In the Name field, type NCOMS.
- In the Host field, type the host name or IP address of the primary ObjectServer, sun22-z01.
- In the Port field, type 4100.
- Click Add.
- Add the Backup ObjectServer server:
- In the Name field, type NCOMS_BKUP.
- In the Host field, type the host name or IP address of the backup ObjectServer, sun25-z00.
- In the Port field, type 4100.
- Click Add.
- Specify the gateway information:
- In the Name field, type NCO_GATE.
- In the Host field, type the host name or IP address of the backup ObjectServer, sun25-z00.
- In the Port field, type 4300.
- Click Add.
- Verify that the ports and names are correct.
- Create NCOMS_BKUP server by running the following command:
nco_dbinit -server NCOMS_BKUP
- Start the object server:
nco_objsvr -name NCOMS_BKUP &
- Configure the gateway NCO_GATE, by copying the following statement:
cp <install_dir>/netcool/omnibus/gates/objectserv_bi/objectserv_bi.props <install_dir>/netcool/omnibus/etc
You are not finished configuring, but you will complete this after you configure the primary object server.
Installing and configuring failover for the primary object server
- On the sun22-z01 server*,* repeat steps 1-5 in section 2.1, which you did for the secondary object server.
Suggestion: Use the same user names and passwords for the primary and secondary object servers.
- Run nco_xigen as you did in configuring the secondary object server.
- There is no need to create the NCOMS object server because this was done by the installer. Start it:
nco_objsrv -name NCOMS &
- Go back to the secondary server, sun25-z00, and start the bi-directional gateway:
nco_g_objserv_bi -name NCO_GATE &
The primary and secondary object servers are now ready. The next step is to install and configure failover for the Network Manager IP data servers.
Installing the primary Network Manager IP data server
- On the sun22-z00 server (see Figure 1), run the installer as the non-root user.
- As you did in section 2.1, select Custom Installation, Multi-server Installation, and Customize settings.
- Select only Install Network Manager core components and Install topology database (MySQL)
- In the object server configuration window, specify the information that you used for the secondary object server.
- Use the following screen captures to guide you:
- As root, follow the directions provided in this last screen capture:
- As the non-root user, start the Network Manager IP data server:
$NCHOME/precision/bin/itnm_control.sh start
- After the installation completes, ensure that the ncp processes are running by doing one of the following steps:
- For UNIX® or Linux® systems: ps -ef | grep ncp
- For Microsoft™ Windows™ systems: Check the services control panel for processes starting ncp_*
Installing the secondary Network Manager IP data server
On the sun21-z01 server*,* repeat steps 2-7 in section 2.3. Use NCOMS as the name of the object server.
Configuring failover for the Network Manager IP data servers.
Define a fixed port for failover
You must change the default setting of DYNAMIC to a fixed port in the ServiceData.cfg file.
- Start virtual domain on the primary server, sun21-z01:
./ncp_virtualdomain -virtualDomain NCOMSV -backupDomain -NCOMS_BKUP -domain NCOMS &
- On the primary data server, sun21-z01, edit the NCHOME/etc/precision/ServiceData.cfg file.
Change DYNAMIC: YES to DYNAMIC: NO in the following line:
SERVICE: NCP_VIRTUALDOMAIN.QUERY DOMAIN: NCOMSV ADDRESS: 9.48.189.79 PORT: 41284 SERVERNAME: sun21-z01 DYNAMIC: YES
SERVICE: NCP_VIRTUALDOMAIN.QUERY DOMAIN: NCOMSV ADDRESS: 9.48.189.79 PORT: 41284 SERVERNAME: sun21-z01 DYNAMIC: NO
- Copy that last line and paste it into the same ServiceData.cfg file on the secondary server (sun25-z00). Save the file.
Configuring Network Manager IP primary data server
- On the sun21-z01server, edit the CtrlServices.cfg file, and insert the following statements:
insert into services.inTray (
serviceName,
servicePath,
domainName,
argList,
dependsOn,
retryCount
)
values
(
"ncp_virtualdomain",
"$PRECISION_HOME/platform/$PLATFORM/bin",
"$PRECISION_DOMAIN",
[ "-domain" , "NCOMS", "-virtualDomain", "NCOMSV", "-backupDomain", "NCOMS_BKUP", "-latency", "60000" , "-debug", "0" ]
[ "ncp_model" ],
5
);
- Save the file.
Configuring the Network Manager IP secondary data server
- On the sun22-z00 server, edit the CtrlServices.cfg file, and insert the following statements:
insert into services.inTray
(
serviceName,
servicePath,
domainName,
argList,
dependsOn,
retryCount
)
values
(
"ncp_poller",
"$PRECISION_HOME/platform/$PLATFORM/bin",
"$PRECISION_DOMAIN",
[ "-domain" , "$PRECISION_DOMAIN" , "-latency" , "60000", "-debug", "0", "-primaryDomain", "NCOMS"],
[ "nco_p_ncpmonitor", "ncp_ncogate", "ncp_mibinit" ],
5
);
insert into services.inTray
(
serviceName,
servicePath,
domainName,
argList,
dependsOn,
retryCount
)
values
(
"ncp_ncogate",
"$PRECISION_HOME/platform/$PLATFORM/bin",
"$PRECISION_DOMAIN",
[ "-domain" , "$PRECISION_DOMAIN", "-server", "NCOMS", "-latency", "60000" , "-debug", "0", "-backup" ],
[ "ncp_f_amos" ],
5
);
insert into services.inTray
(
serviceName,
servicePath,
domainName,
argList,
dependsOn,
retryCount
)
values
(
"ncp_model",
"$PRECISION_HOME/platform/$PLATFORM/bin",
"$PRECISION_DOMAIN",
[ "-domain" , "$PRECISION_DOMAIN" , "-latency" , "60000", "-debug", "0", "-backup" ],
[ "ncp_store", "ncp_class" ],
5
);
insert into services.inTray
(
serviceName,
servicePath,
domainName,
argList,
dependsOn,
retryCount
)
values
(
"ncp_virtualdomain",
"$PRECISION_HOME/platform/$PLATFORM/bin",
"$PRECISION_DOMAIN",
[ "-domain" , "NCOMS_BKUP", "-virtualDomain", "NCOMSV", "-primaryDomain", "NCOMS", "-latency", "60000" , "-debug", "0" ],
[ "ncp_model" ],
5
);
Then, comment out the ncp_disco block:
- Save the file.
- Restart both the primary and secondary Network Manager IP data servers:
NCHOME/precision/bin/itnm_control.sh stop
NCHOME/precision/bin/itnm_control.sh start
Your Network Manager IP servers are enabled for failover.
Installing the secondary Tivoli Integrated Portal server
Important: Be sure you have sufficient space in the /tmp or c:\windows\temp directory (50 000 KB or more).
- On the sun25-z00 server (see Figure 1), run the installer as the non-root user.
- Select Custom Installation, Multi-server Installation, and Customize settings.
- Select *Install user consol*e (Tivoli Integrated Portal component).
- Specify the back up object server information (NCOMS_BKUP).
- Select LDAP.
- Use the following screen captures as guidance:

Tivoli Integrated Portal is installed. Your next procedure is to install the primary Tivoli Integrated Portal server.
Installing the primary Tivoli Integrated Portal server
On the sun22-z01server, repeat steps 2-6 in section 2.6. Use NCOMS as the name of the object server.
Checking the primary and secondary Network Manager IP Tivoli Integrated Portal dashboards
Log into each Tivoli Integrated Portal console as the Tivoli Integrated Portal admin user (shadmin in this case):
Primary: http://sun22-z01.tivlab.austin.ibm.com:16310
Secondary: http://sun25-z00.tivlab.austin.ibm.com:16310
Ensure that the Network Manager IP portals come up and that Network Manager IP discoveries were run.
Install the secondary TBSM data server
- On the sun22-z00 secondary server, as non-root user, start the TBSM installer.
- Select the same 'tivoli' installation location where Network Manager IP is located.
- Click Advanced.
- Select only Data Server (you point to the OMNIbus in a later window):
- For the remaining windows, use the secondary backup object server details (for example, NCOMS_BKUP).
- Create service templates:
"$NCHOME/bin/rad_radshell" < "$NCHOME/guifoundation/webapps/sla/install/BSM_Templates.radsh"
After the installation completes, you can install the primary TBSM data server:
Install primary TBSM data server
On the sun21-z01 server, repeat steps 2-6 in section 2.9. Use the primary object server details, for example, NCOMS.
Install and configure the TBSM XML Tool Kit
- Install the TBSM XMLtoolkit on the primary TBSM data server (sun21-z01), which is located in the TBSM installation image.
- Select CMDB discovery mode. Enter the Tivoli Application Discovery and Dependency Manager (TADDM) configuration parameters (TADDM server host, port, and so on).
- Start the TBSM XMLtoolkit discovery service to start the bulk data pull from TADDM into TBSM. The service runs and periodically runs delta discoveries to get changes in TADDM.
- Repeat steps 1-3 on the secondary TBSM data server, sun22-z00. Select the same TADDM server as in the primary XMLtoolkit.
The following figure illustrates the configuration for the primary and secondary data servers that you just configured:

Configuring High Availability (HA) database for Tivoli Integrated Portal
- If needed, install IBM DB2® Database Version 9.5 on a remote server.
- Create the TIPDB by sourcing the db2profile.
- Run the tipMakedb.bat file or the tipMakedb.sh tipdb file located in the <INSTALL IMAGE>/TBSM/HAScripts directory.
- Run tipdbSchema.bat or tipdbSchema.sh.
Install secondary TBSM Tivoli Integrated Portal dashboard components
- On the server sun25-z00 secondary, as non-root user, start the TBSM installer.
- Choose the same 'tivoli' installation location where Network Manager IP is located.
- Select only Dashboard Server and Load Balancing (HA) features
- Important: For load balancing information, do not select Initialize High Availability Database.
- Complete the database Information:
- Database Hostname (The DB host name)
- Database Port Number (Keep the Default 50000)
- Database User ID (Your DB2 Administrator)
- Database Password (Your DB2 Administrator password)
- Database Name
After the installation completes, you can install the primary Tivoli Integrated Portal dashboard components.
Install primary TBSM Tivoli Integrated Portal dashboard components
- Repeat steps 2-5 on sun22-z01.
- Verify that both Tivoli Integrated Portal server nodes are in the DB2 database:
Log on to your DB system and make sure that you see both server nodes in the NODES table in the tip database.
Enabling server-to-server trust
For the servers to be able to connect to each other and to send notifications server to server, trust must be enabled. This needs to be done on all of the tip servers.
1. Stop the tipservice:
Update the <profile_root>/properties/ssl.client.props file on every node, for example, D:\IBM\tivoli\tip\profiles\TIPProfile\properties.
2. Uncomment the section of the ssl.client.props file that begins with com.ibm.ssl.alias=AnotherSSLSettings, such as:
com.ibm.ssl.alias=AnotherSSLSettings
com.ibm.ssl.protocol=SSL_TLS
com.ibm.ssl.securityLevel=HIGH
com.ibm.ssl.trustManager=IbmX509
com.ibm.ssl.keyManager=IbmX509
com.ibm.ssl.contextProvider=IBMJSSE2
com.ibm.ssl.enableSignerExchangePrompt=true
#com.ibm.ssl.keyStoreClientAlias=default
#com.ibm.ssl.customTrustManagers=
#com.ibm.ssl.customKeyManager=
#com.ibm.ssl.dynamicSelectionInfo=
#com.ibm.ssl.enabledCipherSuites=
3. Uncomment the section of the ssl.client.props file that starts with com.ibm.ssl.trustStoreName=AnotherTrustStore, such as:
com.ibm.ssl.trustStoreName=AnotherTrustStore
com.ibm.ssl.trustStore=${user.root}/etc/trust.p12
com.ibm.ssl.trustStorePassword={xor}CDo9Hgw=
com.ibm.ssl.trustStoreType=PKCS12
com.ibm.ssl.trustStoreProvider=IBMJCE
com.ibm.ssl.trustStoreFileBased=true
4. Update the location of the trust store that the signer should be added to in the com.ibm.ssl.trustStore property of "AnotherTrustStore" by replacing com.ibm.ssl.trustStore=${user.root}/etc/trust.p12 with dircom.ibm.ssl.trustStore=${user.root}/config/cells/TIPCell/nodes/TIPNode/trust.p12
Example: com.ibm.ssl.trustStore=${user.root /config/cells/TIPCell/nodes/TIPNode/trust.p12
5. At the end, the section must look like this:
com.ibm.ssl.trustStoreName=AnotherTrustStore
com.ibm.ssl.trustStore=${user.root}/config/cells/TIPCell/nodes/TIPNode/trust.p12
com.ibm.ssl.trustStorePassword={xor}CDo9Hgw=
com.ibm.ssl.trustStoreType=PKCS12
com.ibm.ssl.trustStoreProvider=IBMJCE
com.ibm.ssl.trustStoreFileBased=true
6. Save the changes made to the ssl.client.props file.
7. Start WebSphere® Application Server.
8. Trust needs to be enabled between all servers in both directions in the server pool, that is, from every server it must be executed to each of the other remote servers. The following command adds a remote server to the local server's trust store.
retrieveSigners.bat NodeDefaultTrustStore AnotherTrustStore -host -port
Example:
On Server 1 you run this script:
retrieveSigners.bat NodeDefaultTrustStore AnotherTrustStore -host server2 -port 16313
You should run this from the D:\IBM\tivoli\tip\profiles\TIPProfile\bin\ directory. The script will show you the following prompt:
Add signer to the trust store now? (y/n)
Type 'y' to add signer to the trust store.
A dialog box is then displayed.
Type in the tipadmin user identity and password and click OK which will dismiss the dialog box. The next prompt will ask if you want to add a signer:
Add signer to the trust store now? (y/n) y
The script will then show you this warning:
A retry of the request might need to occur if the socket times out while waiting for a prompt response. If the retry is required, note that the prompt is not redisplayed if (y) is entered, which indicates the signer has already been added to the trust store.
Then you will see a message like this:
CWPKI0308I: Adding signer alias "default_1" to local keystore
"AnotherTrustStore" with the following SHA digest:
92: DB: D8:64:9C:6C:93:2C:83:BF:00:BE:44:CF:0B:EE:64:81:39:80
And on Server 2 you run this script with similar arguments:
retrieveSigners.bat NodeDefaultTrustStore AnotherTrustStore -host server1 -port 16313
The script will show the following prompt asking if you wish to add a signer to the trust:
Add signer to the trust store now? (y/n) y (type y)
A dialog box is displayed.
Type in the tipadmin user identity and password and click OK.
You will this warning from the script:
A retry of the request might need to occur if the socket times out while waiting for a prompt response. If the retry is required, note that the prompt is not redisplayed if (y) is entered, which indicates the signer has already been added to the trust store.
Then you will see a message like this:
CWPKI0308I: Adding signer alias "default_1" to local keystore
"AnotherTrustStore" with the following SHA digest:
83:2B:C1: DD:C4:11:79:B7:27:2C:EF:0B:69:24:7C:8C:E0:15:09:31
You can find a more detailed tutorial about establishing the server to server trust in the WebSphere Application Server Information Center .
9. Restart the server on each of the systems in the pool.
Checking the primary and secondary TBSM Tivoli Integrated Portal dashboards
Log into each Tivoli Integrated Portal console as the Tivoli Integrated Portal admin user (shadmin in this case):
Primary: http://sun22-z01.tivlab.austin.ibm.com:16310
Secondary: http://sun25-z00.tivlab.austin.ibm.com:16310
Ensure that the TBSM service tree portals come up for both servers.
Configuring failover for TBSM data servers
- Create the failover template:
On either the primary or secondary Tivoli Integrated Portal server (sun25-z00, sun22-z01), run the fo_config command:
<install_dir>/tbsm/bin/fo_config template
The script generates a file with the following name:
<install_dir>/tbsm/bin/fo_config.YYYYMMDDHHMMSS.props
where YYYYMMDDHHMMSS is a time stamp.
You can rename this file to a more convenient name. For this example, perform this step on the tbsmprimary system and rename the output file to fo_config_demo.props.
- Configure the failover template. Using a text editor, open the generated configuration file and modify it as needed:
- Set the value of DATA_PrimaryDataServerHostname to the host name of the primary TBSM data server tbsmprimary.
- Set the value of DATA_PrimaryObjectServerHostname to the host name of the primary ObjectServer tbsmprimary.
- Set the value of DATA_BackupDataServerHostname to the host name of the backup TBSM data server tbsmbackup.
- Set the value of DATA_BackupObjectServerHostname to the host name of the backup ObjectServer tbsmbackup.
- Set the value of DASH_PrimaryDataServerHostname to the host name of the primary TBSM data server tbsmprimary.
- Set the value of DASH_BackupDataServerHostname to the host name of the backup TBSM data server tbsmbackup.
- Copy the fo_config file to each server system:
- Primary TBSM data server (sun21-z01)
- Secondary TBSM data server (sun22-z00)
- Primary Tivoli Integrated Portal server (sun22-z01)
- Secondary Tivoli Integrated Portal server (sun25-z00)
- Run the fon_config command on each server:
- On all systems that are hosting a TBSM data server, run the following command:
<install_dir>/tbsm/bin/fo_config dataserver -Dfo_config.props=<full_path_of_config_file>
For the demo example, run the following command on both tbsmprimary and tbsmbackup
<install_dir>/tbsm/bin/fo_config dataserver -Dfo_config.props=<install_dir>/tbsm/bin/fo_config_demo.props
- On all systems that are hosting a TBSM dashboard server, run the command:
<install_dir>/tbsm/bin/fo_config dashboardserver -Dfo_config.props=<full_path_of_config_file>
For the demo example, run the following command on both tbsmprimary and tbsmbackup
<install_dir>/tbsm/bin/fo_config dashboardserver -Dfo_config.props=<install_dir>/tbsm/bin/fo_config_demo.props
Starting the servers
To start all the servers in a server failover configuration, you need to start the following processes in this order:
- ObjectServer (primary)
- ObjectServer (secondary)
- OMNIbus gateway (secondary)
- TBSM Data Server (primary)
- TBSM Data Server(secondary)
- Tivoli Integrated Portal servers (primary)
- Tivoli Integrated Portal server (secondary)
Testing Failover and Tivoli Integrated Portal Availability
Generally for testing failover, you want to stop the primary server and ensure availability of service through the secondary or backup server. The following sections describe scenarios on how to test failover for TBSM, Network Manager IP and Tivoli Integrated Portal server availability.
Testing TBSM failover
Scenario 1: TBSM data server availability
- Stop the primary TBSM data server (for example, sun21_z01) by stopping the TBSM WebSphere Application Server server.
- Log into the Tivoli Integrated Portal console and navigate to the TBSM service tree.
- Ensure that the service tree data is available and up-to-date.
- Log out of the Tivoli Integrated Portal console.
- Start the primary TBSM data server.
- Log into the Tivoli Integrated Portal console and navigate to the TBSM service tree.
- Ensure that the service tree data is available and up-to-date.
Scenario 2: Object server (OMNIbus) availability
- Stop the primary Object server (for example, sun22_z01) by stopping the nco process.
- Log into the Tivoli Integrated Portal console and navigate to the TBSM service tree.
- Ensure that the service tree data is available and up-to-date.
- Log out of the Tivoli Integrated Portal console.
- Start the primary Object server.
- Log into the Tivoli Integrated Portal console and navigate to the TBSM service tree.
- Ensure that the service tree data is available and up-to-date.
Scenario 3: TBSM data server and object server availability
- Stop the primary Object server and primary TBSM data server by stopping the nco process.
- Log into the Tivoli Integrated Portal console and navigate to the TBSM service tree.
- Ensure that the service tree data is available and up-to-date.
- Log out of the Tivoli Integrated Portal console.
- Start the primary Object server and the primary TBSM data server.
- Log into the Tivoli Integrated Portal console and navigate to the TBSM service tree.
- Ensure that the service tree data is available and up-to-date.
Testing Network Manager IP failover
Scenario 1: Network Manager IP data server availability
- Stop the primary Network Manager IP data server (for example, sun21_z01) by stopping ncp_ctrl process.
- Log into the Tivoli Integrated Portal console and navigate to the Network Manager IP Discovery Status portal.
- Ensure that the portal data is available and up-to-date.
- Log out of the Tivoli Integrated Portal console.
- Start the primary Network Manager IP data server.
- Log into the Tivoli Integrated Portal console and navigate to the ITNM Discovery Status portal.
- Ensure that the portal data is available and up-to-date.
Scenario 2: Object server (OMNIbus) availability
- Stop the primary Object server (for example, sun22_z01) by stopping the nco process.
- Log into the Tivoli Integrated Portal console and navigate to the Network Manager IP Discovery Status portal.
- Ensure that the portal data is available and up-to-date.
- Log out of the Tivoli Integrated Portal console.
- Start the primary Object server.
- Log into the Tivoli Integrated Portal console and navigate to the Network Manager Discovery Status portal.
- Ensure that the portal data is available and up-to-date.
Scenario 3: Network Manager IP data server and object server availability
- Stop the primary Object server and the primary Network Manager IP data server by stopping the nco process.
- Log into the Tivoli Integrated Portal console and navigate to the Network Manager IP Discovery Status portal.
- Ensure that the portal data is available and up-to-date.
- Log out of the Tivoli Integrated Portal console.
- Start the primary Object server and the primary Network Manager IP data server.
- Log into the Tivoli Integrated Portal console and navigate to the Network Manager Discovery Status portal.
- Ensure that the portal data is available and up-to-date.
Testing Tivoli Integrated Portal High Availability
Scenario 1: Tivoli Integrated Portal server availability
- Stop the primary Tivoli Integrated Portal server by stopping the Tivoli Integrated Portal WebSphere Application Server server (WAS profile TIPProfile).
- Log into the Tivoli Integrated Portal console as an LDAP user and navigate to any data portal (for example, TBSM service tree).
- Ensure that you are able to log in as an LDAP user and the portal data is available.
- Log out of the Tivoli Integrated Portal console.
- Start the primary Tivoli Integrated Portal server.
- Log into the Tivoli Integrated Portal console as an LDAP user and navigate to any data portal.
- Ensure that you are able to log in and that portal data is available.
|