I have a configuration that has 8 gpfs servers. The servers each have two local paths to the storage. One is an optimal path, the other is less efficient because it must pass data through the peer controller. I had hoped that using the useNSDserver=always mount option would eliminate the use of the non-optimal path. What I have found is that it only seems to work if I reduce the server list for the NSDs to a single server.
For example, mounted using no options (asneeded, is the default according to the docs), and a single server in the NSD server list, I still use the correct path from half of the IOs but the incorrect path for the other half. If I mount with -o useNSDservers=always option, it works and gives me improved performance. The problem is that with only a single server listed in the server list, there is no resiliency in the event one of the servers were to go down. When I add additional hosts to the server list, it appears that it is still doing local discovery of disks even with the -o useNSDservers=always mount option.
Also, I should clarify that all IO is being performed from the servers. This particular configuration does not use any GPFS clients.
How can I disable local disk discovery while also including multiple servers in the nsd server list?