This topic has been locked.
4 replies Latest Post - 2013-03-28T09:44:27Z by SystemAdmin
Pinned topic problem with multi-filesytem cluster
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
we have a cluster with several filesystems, served by the same set of NSD servers. We are observing the following problem. When the disk backend of one of the filesystems saturates, we start observing large NSD thread I/O waiters on the NSD servers, and this is expected. Unfortunately, this ends up to consume the total number of NSD threads (nsdMaxWorkerThreads 256), and so all the operations of the other filesystems are delayed as well, since all NSD I/O threads are busy. Is there any smart way to handle this, apart from the obvious solution of using separate NSD's for different filesystems?
Updated on 2013-03-28T09:44:27Z at 2013-03-28T09:44:27Z by SystemAdmin
bdherr 060000C2N29 PostsACCEPTED ANSWER
Re: problem with multi-filesytem cluster2013-03-26T14:53:13Z in response to VincenzoVagnoniWhat version of GPFS are you running? You could throttle down the clients by lowering the maxMBpS or lowering prefetchThreads, BUT this all depends on your workload and I/O access paterns and may have other unintended consequences.
VincenzoVagnoni 27000328NS94 PostsACCEPTED ANSWER
Re: problem with multi-filesytem cluster2013-03-27T10:21:35Z in response to bdherrthe GPFS version is 3.4.0-17. Lowering maxMBps or prefetch threads cannot be specified at filesystem level as far as I know, how could it help? We have thousands of computing clients and tens of different filesystems mounted on them. I don't think that what you suggest can be of any use. If there is no way to limit the number of NSD I/O threads per filesystem on a NSD server, it means that if a filesystem is saturating the backend storage, this has a huge impact on all other filesystems served by the same NSD server. I.e., one should avoid using the same NSD servers for different filesystems. But I have never read this in the GPFS documentation, nor in best practices and tutorials. Too bad...
bdherr 060000C2N29 Posts
SystemAdmin 110000D4XK2092 PostsACCEPTED ANSWER
Re: problem with multi-filesytem cluster2013-03-28T09:44:27Z in response to bdherrSure. Dont forget something.
A clustered file system is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system (only direct attached storage for each node). Clustered file systems can provide features like location-independent addressing and redundancy which improve reliability or reduce the complexity of the other parts of the cluster. Parallel file systems are a type of clustered file system that spread data across multiple storage nodes, usually for redundancy or performance.