Testing Virtual IP and failovers

When you are managing an SMB environment that is integrated with IBM Storage Ceph, troubleshooting the Clustered Trivial Database (CTDB) configuration can ensure high availability and seamless failover functions. Learn to verify CTDB configuration, identifying common issues, and performing both real-world and simulated failover tests.

About this task

Proper configuration and network setup are critical for ensuring optimal performance during failover scenarios. Network separation is one of the CTDB best practices.

Network segregation helps ensure that the network used by the SMB clients to access CephFS volumes that are exported by the SMB service is separate from the internal Ceph network. Failing to do leads to performance degradation, especially during failovers of CTDB between nodes. This network segregation results in longer failover times and reduced performance when accessing SMB shares from Windows.

With this procedure you can verify the CTDB configuration.

Note: Run the CTDB commands inside the ctdb container.

Procedure

  1. Verify a real-world failover scenario.
    1. You can verify the virtual IP (VIP) addresses of an offline server or the server put under maintenance are carried over to another server in the cluster.
      Verify that CTDB is running by using the following commands. 
      ctdb status
ctdb ping -n all
    2. Run the ctdb ip command to view the list of public IP addresses and the node that is hosting them.
      A value of -1 for a node number indicates that an address is not currently hosted.
    3. To view all VIPs across all nodes in the cluster, run the ctdb ip all command to show addresses defined on all nodes.
    4. Mount an SMB share by using any of the VIPs to check that the VIP is being assigned properly.
    5. Run ctdb ip to locate the physical node serving the VIP.
    6. Shut down or put the node under maintenance to verify successful migration of VIPs.
      Note: When the server serving the VIP is shut down, the I/Os pause for a few seconds and then resume.
  2. Verify by failover simulation.
    1. Verify that CTDB is running by using the following commands. 
      ctdb status
ctdb ping -n all
    2. Run the ctdb ip command to view the list of public IP addresses and the node that is hosting them.

      A value of -1 for a node number indicates that an address is not currently hosted.

      Note: This command only shows public IP addresses that are defined on the current node.
    3. If different groups of public IP addresses are defined on different nodes, use ctdb ip all to show addresses defined on all nodes.
    4. Mount an SMB share by using any one of the VIPs.
    5. Run the ctdb ip command to locate the physical node serving the VIP.
    6. Run the ctdb disable command to verify successful migration of VIPs.
      Note: When the server serving the VIP is disabled through the command-line interface, the I/Os pause for a few seconds and then resume.