Validating Cassandra and ZooKeeper installation

After you install and start all the Cassandra and ZooKeeper nodes, you can validate the installation to ensure all the nodes are up and running and can communicate with each other.

1. You can validate Cassandra installation by one of the following ways or can try all:
   1. Check the status of the Cassandra nodes in your cluster - Go to the /<Install_Dir>/apache-cassandra/bin/ directory and type the ./nodetool status command. If the status for all the nodes shows as UN, then the nodes are up and running. If the status for any node shows as DN, then that particular node is down.
      Tip: To run nodetool, JAVA_HOME must be set to the location of IBM JDK 8.

      When a node is down, it usually means that the node did not start properly. It can take several minutes for the node to start. Ensure the node is running or there are no network problems preventing the node from communicating with other nodes in the cluster. Run ./nodetool status again in a few minutes until the node is in UN status.

   2. Connect to a Cassandra node and run a query - Go to /<Install_Dir>/apache-cassandra/bin/ directory and type ./cqlsh <host> <port>, where <host> is the host name or IP address of any server where a Cassandra node is installed. <port> is optional if you used the default port during installation.
   3. Verify Cassandra log files - Go to /<Install_Dir>/apache-cassandra/logs/system.log.

      The following line means that Cassandra node can listen for connections on port 9160:

      INFO [main] <timestamp>,941 ThriftServer.java (line 118) Binding thrift service to <host name>/<IP address>:9160

      The following line means the node is marked up:

      INFO [StorageServiceShutdownHook] <timestamp>,376 StorageService.java (line 1530) Node <host name>/<IP address> state jump to normal

      Errors usually have "ERROR" in the first column.

2. You can validate ZooKeeper installation by one of the following ways or can try all:
   1. Check that the ZooKeeper ensemble is working - telnet <zk host><port> and type ruok.
   2. Verify ZooKeeper and ZooKeeper watchdog log files - Go to /<Install_Dir>/zookeeper/bin/zookeeper.out and /<Install_Dir>/zookeeper/watchdog/watchdog.log.X.

      When the ZooKeeper watchdog starts, you will see the following message:

      com.ibm.mailbox.zkwatchdog.ZKWatchDog <init>

      ZooKeeper watchdog connects to other ZooKeeper nodes, and when it does, the following is displayed in the log file:

      com.ibm.mailbox.zkwatchdog.ZKWatchDog <init>

      INFO: ZooKeeper server: id=<ID number>, host=<host name or IP address>

      If the watchdog cannot connect to a ZooKeeper server, the following is displayed:

      WARNING: ZooKeeper server at <hostname>:<portnumber> is offline