Technical Blog Post
Abstract
tb_break tool to check CM tiebreaker for purescale
Body
When a CM tiebreaker is required for purescale, tb_break tool from RSCT could be provided for manual test if a tiebreaker is working as expected.
You can check the detail in the following link.
This post is especially about the tb_break usage for purescale CM tiebreaker.
Firstly please check the command syntax in detail.
----------------------------------------
/usr/sbin/rsct/bin/tb_break syntax
usage: ./tb_break [-h] [-v] [-w <secs>] [ -i | -b | -f | -l | -u | -p ] [-t tb-type] <dev-spec>
-h show this help
-v verbose
-i info about the tie-breaker (currently only WWID on Linux)
-f reserve the tie breaker after breaking it
-l reserve tb (no break)
-u release tb (no break)
-p preempt tb (replace SCSIPR reserve)
-w <secs> wait for a while before close/exit
-t <type> "DISK"
<dev-spec> DEVICE=<device name>
PVID=<pvid>
e.g., ./tb_break -t DISK "DEVICE=/dev/hdisk0"
./tb_break -t DISK "PVID=0024a09a08f1879d"
./tb_break -t DISK "UUID=6960c14b89eb458d4e04b08676463c00"
-t <type> "EXEC"
<dev-spec> PATHNAME=absolutePath
e.g., ./tb_break -t EXEC "PATHNAME=/opt/rsct/bin/tbScript"
----------------------------------------
We can follow the step below for the test.
1. Make sure that db2 is stopped on all nodes.
2. (As root) On one of the nodes in the cluster (say node A), run /usr/sbin/rsct/bin/tb_break -v -l -t DISK "PVID=<pvidID>". This will reserve the disk for this node.
(To get pvid, please use the lspv command.)
3. (As root) On another node in the cluster (say node B), run the same command /usr/sbin/rsct/bin/tb_break -v -l -t DISK "PVID=<pvidID>". This should fail since the disk is already reserved by node A.
4. Now, release the disk from node A by running /usr/sbin/rsct/bin/tb_break -v -u -t DISK "PVID=<pvidID>" as root.
The following is the example from the problem case.
----------------------------------------
First, reserving the tiebreaker from Host 1
/usr/sbin/rsct/bin/tb_break -l -t SCSIPR DISK "PVID=00f90f31db5d94ca"
Initializing SCSIPR tie-breaker (DISK PVID=00f90f31db5d94ca)
SCSIPR_init Entered
FOUND: PVID=00f90f31db5d94ca as DISK=/dev/hdisk6
Command(/usr/sbin/rsct/bin/lsnodeid) returns exitCode=0
PR Key obtained from lsnodeid : 6697552216055880078
SCSIPR_PRKEY: lsnodeid(rc=0): 6697552216055880078
SCSIPR_init: Leaving registered /dev/hdisk6, prkey=6697552216055880078
Reserving tie-breaker (DISK PVID=00f90f31db5d94ca)
SCSIPR_reserve: dev=/dev/hdisk6, prkey=6697552216055880078, result=0, status=0
tb_reserve status GRANTED(0) (errno=0)
Reservation is successful
Reserving the tiebreaker from Host 2 fails due to reservation from host1
/usr/sbin/rsct/bin/tb_break -l -t SCSIPR DISK "PVID=00f90f31db5d94ca"
Initializing SCSIPR tie-breaker (DISK PVID=00f90f31db5d94ca)
SCSIPR_init Entered
FOUND: PVID=00f90f31db5d94ca as DISK=/dev/hdisk6
Command(/usr/sbin/rsct/bin/lsnodeid) returns exitCode=0
PR Key obtained from lsnodeid : 393486876566146278
SCSIPR_PRKEY: lsnodeid(rc=0): 393486876566146278
SCSIPR_init: Leaving registered /dev/hdisk6, prkey=393486876566146278
Reserving tie-breaker (DISK PVID=00f90f31db5d94ca)
Failed to openx(/dev/hdisk6,xflags=0x2) for persistent reservation. errno = 16
SCSIPR_reserve: dev=/dev/hdisk6, prkey=393486876566146278, result=16, status=1
tb_reserve status DENIED(1) (errno=16)
----------------------------------------
NOTES
1. Make sure you release the disk at the end of the tests. Else next time, in a real failure event, the cluster might not be able to acquire the tiebreaker.
2. Perform these tests when the entire cluster is offline.
3. To get <pvidID> on AIX, you can run lspv
For example, consider the following:
$ lspv
hdisk2 00cea03ef81422bb None
hdisk3 00cea03ef814e021 None
hdisk4 00cea03ef81504ff None
hdisk5 00cea03ef8152967 None
hdisk6 00cea03ef8154fb5 gpfs1nsd
hdisk7 00cea03ea3a0bdbd rootvg active
hdisk8 00cea03ea3b2ff94 homevg active
My tiebreaker disk is hdisk2. The <pvidID> is 00cea03ef81422bb.
[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
UID
ibm11140538