when applying a list policy using the command:
mmapplypolicy gpfs_lhcb -P yamssListPolicy -B 100000 -N nodeA,nodeB -M YAMSSDEVICENAME=gpfs_lhcb -s /var/tmp
where yamssListPolicy is:
RULE EXTERNAL LIST 'LIST TAPE FILES' EXEC '/usr/local/yamss/bin/yamssProcessList' OPTS 'tape YAMSSDEVICENAME'
RULE 'list tape files' LIST 'LIST TAPE FILES' SHOW(VARCHAR(FILE_SIZE)||' '||VARCHAR(KB_ALLOCATED)) WHERE MISC_ATTRIBUTES LIKE '%M%'
RULE EXTERNAL LIST 'LIST ALL FILES' EXEC '/usr/local/yamss/bin/yamssProcessList' OPTS 'all YAMSSDEVICENAME'
RULE 'list all files' LIST 'LIST ALL FILES'
I'm getting a very high CPU load of mmfsd, i.e. about 400%, and the machine where mmapplypolicy is issued is completely frozen for several minutes until the first phase of the scan ends and the CPU load goes down. There is plenty of memory available. Can you tell me what's wrong with this policy? GPFS is 3.4.0-17 on kernel 2.6.18-194.17.1.el5 x86_64
I did not notice this behaviour in the past with GPFS 3.3 series.
This topic has been locked.
3 replies Latest Post - 2012-12-23T02:41:50Z by SystemAdmin
Pinned topic very high load of mmfsd with mmapplypolicy and list scan
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
Updated on 2012-12-23T02:41:50Z at 2012-12-23T02:41:50Z by SystemAdmin
FredStockatIBM 120000FABX42 PostsACCEPTED ANSWER
Re: very high load of mmfsd with mmapplypolicy and list scan2012-12-22T15:35:14Z in response to VincenzoVagnoniDo you have information as to the time it took for this policy to run on GPFS 3.3 versus GPFS 3.4? And presumably GPFS 3.3 was run on the identical hardware configuration but please confirm that is true.
VincenzoVagnoni 27000328NS108 PostsACCEPTED ANSWER
Re: very high load of mmfsd with mmapplypolicy and list scan2012-12-22T15:53:17Z in response to VincenzoVagnonirunning on 3.3 takes much longer. I have seen that tweaking -n option down from the default (24) value the load of mmfsd goes down and it takes longer. So I guess that if we want a faster scan we need to use HW with more CPU cores
SystemAdmin 110000D4XK2092 PostsACCEPTED ANSWER
Re: very high load of mmfsd with mmapplypolicy and list scan2012-12-23T02:41:50Z in response to VincenzoVagnoniWell, that was the point of the question. If 3.4 is "burning" more CPU cycles, perhaps it's getting more scanning done per second and finishing faster.
Be that as it may, we are very aware of these kinds of problems and are hard at work on a much better way to run "maintenance" jobs like policy and restripe, in a way that will have minimal impact on other jobs/processes/tasks that are using the file system.