What happens to the order of NSDs in the file system after a mmrpldisk? None of the examples in the manuals show the output of "mmlsnsd" before and after the mmrpldisk command.
original NSD order for GPFS file system "fs1". Assume the "A" NSDs are on storage array A, and the "B" NSDs are on storage array B
nsd_A1, nsd_B1, nsd_A2, nsd_B2
This configuration is well balanced, toggling between storage array A and storage array B for each sequential NSD.
Replace nsd_B1 with new nsd_B3
mmrpldisk fs1 nsd_B1 -F stanza_file_for_nsd_B3
What is the NSD ordering after the replace? Is it:
nsd_A1, nsd_B3, nsd_A2, nsd_B2 (the new NSD replaces the original NSD in the ordered list)
nsd_A1, nsd_A2, nsd_B2, nsd_B3 (equivalent to order obtained by mmdeldisk followed by mmadddisk)
The documentation indicates that mmrpldisk is logically similar to a mmdeldisk followed by mmadddisk, but more efficient because original NSD's contents are copied to the new NSD and file system balance is maintained (assuming the new NSD is the same size, data/metadata Usage type, and storage pool assignment). As such, a file system rebalance is not needed.
However, the documentation is unclear if the ORDERING of the NSDs in the file system is maintained, which can effect IO balance across storage arrays. In the example above, the first case, the new NSD replaces the original NSD, while keeping the NSD ordering. The file system is still well balanced across storage arrays, toggling between storage array A and storage array B (as the original configuration)
In the second case, the original NSD was effectively deleted from the NSD list, and the new NSD was appended to the tail of the list. In the example, now the NSD ordering is NOT well balanced across two storage arrays, with two adjacent IOs likely being sent to the different LUNs, but on the same storage array, resulting in higher congestion on the array, and a micro-clumping IO behavior. Yes, this is a fine point, but it can and does make a difference.
So, in summary, does the mmrpldisk command maintain the NSD ordering? If not, the administrator may want to make additional adjustments.
Thank you for your help.