I got a general question:
are there any fundamental differences in client performance and system stability between
having a large number of client nodes (a few thousand) together with the NSD servers in one big GPFS cluster OTOH and having the clients in several smaller clusters mounting the file systems remotely from one storage cluster comprisiung the NSD servers?
In particular I was told by someone that the "all-in-one" solution would yield a much better performance in HPC environments than using remotely mounting client clusters. However, the explanation why remained vague.
Is there a general difference in accessing data and metadata from within a cluster and from another, remotely mounting one? What are the benefits of these approaches vs the respective other one?
I could imagine that within a cluster one has all-to-all connections right from the start, whereas connections to remote cluster nodes will be opened as required only.
Thank you in advance for your efforts
Pinned topic Remote cluster vs. Local Cluster performance- advantages and disadvantages
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
Updated on 2012-09-08T09:29:56Z at 2012-09-08T09:29:56Z by ufa
SystemAdmin 110000D4XK2092 Posts
Re: Remote cluster vs. Local Cluster performance- advantages and disadvantages2012-09-06T19:25:07ZThis is the accepted answer. This is the accepted answer.Hi Uwe,
I can not vouch for the GPFS internals or any particular performance. But the standard setup in a Bluegene supercomputer environment is with dedicated GPFS clusters for the storage (NSDs) and a separate cluster for the GPFS clients (Bluegene IO-nodes). At least in terms on bandwidth there is no disadvantage to this. We have done some tuning work and we able to achieve application performance very close to the hardware/wire speed.
If there is an advantage for you to split your GPFS in several independent clusters, then I would not hesitate to do that.
SystemAdmin 110000D4XK2092 Posts
Re: Remote cluster vs. Local Cluster performance- advantages and disadvantages2012-09-06T21:05:07ZThis is the accepted answer. This is the accepted answer.There's no substantial difference in regular IO performance for remote vs local cluster nodes when only two clusters are involved. When two (or more) remote clusters access the same subset of files, things get more complicated, because GPFS has an obligation to avoid direct network communication between nodes in different remote clusters, but this still only has a minimal impact on IO performance. Once a remote node joins the home cluster, by the virtue of mounting a remote file system, it's treated like a regular node for normal IO purposes. What is different for remote clusters is administration -- there are separate clusters that need to be managed independently. Also, the set of nodes to use as workers for distributed admin commands, e.g. mmrestripefs, is restricted to local cluster nodes.
ufa 0600026VN6166 Posts
Re: Remote cluster vs. Local Cluster performance- advantages and disadvantages2012-09-08T09:29:56ZThis is the accepted answer. This is the accepted answer.Thank you for your replies. That supports what we plan to do, however, if somebody argues bluntly he has seen contradicting scenarios one should at least ask, so forgive my ignorance.