IBM Streams 4.2

Upgrading an external ZooKeeper server

Use this procedure to upgrade the external ZooKeeper servers that are used by IBM® Streams enterprise domains.

About this task

The upgrade procedure that you use depends on how your ZooKeeper ensemble is configured.

Rolling upgrade: If each ZooKeeper server is running on a dedicated machine and the hostname or zkconnect string is not changing, you can upgrade the server while the IBM Streams domains are running.
Standard upgrade: If the ZooKeeper servers share a machine or there are changes to the hostname or zkconnect string, you must perform a standard upgrade on each server. IBM Streams domains must be stopped before performing the upgrade.

Procedure

For a rolling upgrade, complete the following steps:
1. Stop any ZooKeeper watchdog or monitor that restarts the ZooKeeper server.
2. Verify that the ZooKeeper ensemble is up and running by using the ZooKeeper srvr or stat command. For a srvr command example, see the ZooKeeper setup procedure.
3. Stop one of the ZooKeeper servers that is defined in the zoo.cfg file and verify that a quorum still exists.
  In the following example, the ZooKeeper ensemble includes zkserver1, zkserver2 and zkserver3. The client port number is 2181.
  1. To stop zkserver1, enter the following command on zkserver1:
    ZooKeeper-installation-directory/bin/zkServer.sh stop
  2. To verify that a quorum still exists, use the srvr command and specify another server in the ZooKeeper ensemble on the command, for example:
    echo srvr | nc zkserver2 2181
    
    If a quorum exists, the srvr command output includes information about the ZooKeeper version, latency, and other information as shown in the following example:
    Zookeeper version: 3.4.6-1569965, built on 08/11/2016 09:09 GMT Latency min/avg/max: 0/4/2834 Received: 14529333776 Sent: 14758752055 Connections: 1616 Outstanding: 4 Zxid: 0x952925303 Mode: follower Node count: 237674
4. Back up the following directories on the server:
  - dataDir
  - dataLogDir, if specified in the zoo.cfg file
5. Perform the upgrade on the server.
6. Copy the content in the following directories to the upgraded server:
  - dataDir
  - dataLogDir, if specified in the zoo.cfg file
7. If applicable, transfer the following files to the upgraded server:
  - zoo.cfg
  - java.env
  - log4j.properties
8. Verify that the zoo.cfg file references the correct file path for the dataDir and dataLogDir directories, and that the myid files are configured correctly.
9. Restart the upgraded ZooKeeper server, for example:
```
ZooKeeper-installation-directory/bin/zkServer.sh start
```
10. Repeat Steps 2-9 for the other ZooKeeper servers in the ensemble.
11. After all servers in the ensemble are upgraded, restart the ZooKeeper watchdog or monitor, if applicable.
For a standard upgrade, complete the following steps:
1. Stop the IBM Streams domains that are using the ZooKeeper ensemble. To stop a domain, enter the following command:
```
streamtool stopdomain -d domain-id
```
2. Log in as user root on each IBM Streams resource that is running the domain controller service as a Linux system service, and enter the following command:
```
streamtool stopdomainhost -d domain-id
```
3. Stop any ZooKeeper watchdog or monitor that restarts the ZooKeeper server.
4. Stop the ZooKeeper servers. To stop a ZooKeeper server, enter the following command on the server:
```
ZooKeeper-installation-directory/bin/zkServer.sh stop
```
5. Back up the following directories on each server:
  - dataDir
  - dataLogDir, if specified in the zoo.cfg file
6. Perform the upgrade on each server.
7. If applicable, transfer the following files to each server:
  - zoo.cfg
  - java.env
  - log4j.properties
8. Verify that the zoo.cfg file references the correct file path for the dataDir and dataLogDir directories, and that the myid files are configured correctly for each server.
9. Copy the content in the following directories to each server:
  - dataDir
  - dataLogDir, if specified in the zoo.cfg file
10. Restart each of the upgraded ZooKeeper servers. To start a ZooKeeper server, enter the following command on the server:
```
ZooKeeper-installation-directory/bin/zkServer.sh start
```
  When a quorum is formed, the data are automatically replicated to the servers in the quorum.
11. Verify that the ZooKeeper ensemble is up and running by using the ZooKeeper srvr or stat command. For a srvr command example, see the ZooKeeper setup procedure.
12. After all servers in the ensemble are upgraded, restart the ZooKeeper watchdog or monitor, if applicable.
13. Log in as user root on each IBM Streams resource that is running the domain controller service as a Linux system service, and complete the following steps for each domain:
  1. Update the zkconnect property in the following file to the proper value:
    /etc/InfoSphere_Streams/domain-id/controller.properties
  2. Start the domain controller service on the resource by entering the following command:
    streamtool startdomainhost -d domain-id
14. Start the IBM Streams domains. To start a domain, enter the following command:
```
streamtool startdomain -d domain-id
```