Which command can clear gpfs cache?
I have to shutdown and start gpfs to clear cache. this sulution is troublesome and time consuming.
Any cammand can do this?
This topic has been locked.
14 replies Latest Post - 2013-01-21T09:56:52Z by ValerieR
Pinned topic how to clear gpfs cache manually?
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
Updated on 2013-01-21T09:56:52Z at 2013-01-21T09:56:52Z by ValerieR
dlmcnabb 120000P4JT1012 PostsACCEPTED ANSWER
Re: how to clear gpfs cache manually?2012-12-11T21:43:26Z in response to SystemAdminmmfsctl $fsname suspend/resume will only flush the dirty data. It will not empty the cache. Unfortunately, the only way to clear the cache is to unmount/mount the filesystem. And even this will not do anything on any "manager" node that are handling tokens for the filesystem.
Re: how to clear gpfs cache manually?2012-12-12T05:55:37Z in response to SystemAdminI am doing some full-text search tests on gpfs.
The full-text search engine has many parameters which need to configured.
I need to test which parameters have the best performance.
In test, I find gpfs cache will affect the program.
when i run a test case, I must clear gpfs cache.
otherwise，when I try a different parameters configuration, and rerun the test case again.
it will have a better performance than previous one.
So I must clear gpfs cache before rerun the test.
sberman 27000268QN59 PostsACCEPTED ANSWER
Re: how to clear gpfs cache manually?2012-12-12T13:59:22Z in response to SystemAdminIf this is for test and performance analysis only, not a regular production operation, as Dan said already, the only way to clear all cached data is to unmount the GPFS file system. mmunmount and then mmmount it back again. You will have no cached data left over, but all access to the file system will be interrupted.
HajoEhlers 0100001U0A251 PostsACCEPTED ANSWER
Re: how to clear gpfs cache manually?2012-12-12T15:30:42Z in response to SystemAdminYou have some other options as well depending want you looking for.
1 ) Clear the GPFS pagepool cache ( Metadata is still cached ) - by making it invalid.
Note: I assume that the seqDiscardThreshold can be dynamicly changed. If not i would suggest to set the seqDiscardThreshold to the same size as the pagepool.
Set seqDiscardThreshold to the size of the pagepool Write/read a file in the size of the pagepool reset seqDiscardThreshold to its original value or in case the pagepool is to large: decrease the pagepool to a small value ( 64MB) Set seqDiscardThreshold to the size of the pagepool Write/read a file in the size of the pagepool reset the pagepool to its original value. reset seqDiscardThreshold to its original value
This should overwrite everything in the pagepool
2) Use directo i/o to bypass GPFS cache.
Use the mmchattr -D to set direct i/o flag for all files which shall not be cached.
All not tested
Re: how to clear gpfs cache manually?2012-12-12T20:16:04Z in response to HajoEhlersYour safest option which is only a few seconds of interruption is to
mmmount xyz -a (if you don't have automount set)
Shutting down the daemon kills the mmfsd (GPFS) process on each node, so for sure, you will be starting "clean" wrt GPFS.
If you just unmount, then you can't be sure that there is no buffer/memory "leak" that is accumulating between tests.
No, I'm not saying mmfsd has memory leasks, but I couldn't guaranteee 100% that it doesn't either!
Re: how to clear gpfs cache manually?2012-12-26T16:15:03Z in response to SystemAdminHello guys
I am also interested by this question about flushing or clear the gpfs cache.
My problem is in another context (production).
For backup purpose, we use a IBM flashcopy like process to copy GPFS data on a disaster recovery/backup server, thru a Storage Array disk copy mechanism(Business Copy on XP24000 Storage Array). This work correctly except for one file, that is created just before the split of the copy (this file is used to open the Oracle database in recover mode). From time to time (not every day), this file appear to be empty (size 0) on the backup server.
On the source (production) server, this file is not empty. It's a very little file in size (87 bytes).
We make multiple searches to try to understand why this file was sometimes empty. One of the supposition is that this file has not been flushed from the GPFS cache before the split command.
In the context of our customer, it's not possible to unmount the GPFS FS (Oracle RAC with SAP application), as we do online backup. So one of my idea was to use the Direct I/O attribute on this file.
As I read before in this post, do you think it's a good idea and that it could solve my problem ?
Does somebody already use this attribute and in what case?
Thanks a lot for your help
Re: how to clear gpfs cache manually?2012-12-26T16:29:43Z in response to ValerieRValerieR,
Are you using mmfsctl fsname suspend; <flash-copy!!>; mmfsname resume;
That should work. If not, there is a bug somewhere.
I believe just about any other sequence you try will have a window of vulnerability.
FYI, small files that will fit in an inode, are stored in the inode with the latest release of GPFS.
Re: how to clear gpfs cache manually?2012-12-27T08:09:43Z in response to SystemAdminHello
No, we don't use the mmfsctl suspend / resume commands at this time. It was a decision validated with IBM to not use this command, to avoid loss of application IO during this period.
If we have a guaranty that when you do a suspend, the IO are kept somewhere in memory but not loss, we can try to use this command, but I saw anything about this in the documentation I read and i'm not sure that we don't loose any IO. What happend exactly when you use the mmfsctl suspend command ? Does it make some retry, does it generate some system errors ?
I understand that mmfsctl suspend command will be the best solution for my problem, but if we still not want to use these commands, can we suppose that the DIO attribute could solve our problem ?
We are at this time in release 22.214.171.124 of GPFS. As it take a long time to be able to migrate thru a new release of GPFS because of a production environment, we cannot go into a new release before the end of 2013 year !!! But we are going to prepare during the beginning of 2013 the future migration to 3.5 release.
Thanks for your help
Re: how to clear gpfs cache manually?2012-12-27T16:20:25Z in response to ValerieRWell that was my simple q&a....
Now let's step back and see what you are really trying to accomplish!
You have a GPFS filesystem, and an Oracle database, (anything else? !!)
In theory, I presume, everything is programmed and set up so you would "survive" a sudden power outage followed by restart. Yes?
That is usually done via a combination of techniques, including data base logging, file system logging, file system fsync calls,
NVRAM in your disk controllers, etc, etc. In short, at any instant, the state of your system, even excluding "volatile" memory buffers, is spread over several components and subsystems!
So, unless you've thought all that through, and can prove that your flash copy is capturing everything it needs to capture -
you'd be fooled to think that the flash copy is sufficient for recovery purposes...
If you really want to continue down this line, you've got to be sure that everything is going to the drives that are being flash copied, and that the flash copy is truly a complete "point-in-time" copy.
Then you have to come up with a recovery procedure, which would probably be - a complete power-off shutdown of all relevant systems,
then bring up your disk drive system; restore it to the point-in-time. Then bring up the filesystem, let it run through its own log recovery procedures (GPFS will do that before it declares a mmmount successful.) Then bring up the database, let it run through its log recovery, etc, etc.
Good luck! This is not easy stuff. Don't let any sales rep type tell you differently! (IBM, Oracle or otherwise ;-) !! )
Re: how to clear gpfs cache manually?2013-01-02T05:58:12Z in response to ValerieRI'm not sure what "loss of application IO" means. It appears that the fear is that after "mmfsctl suspend" is issued, some IOs could be failed or discarded. This most certainly won't happen -- a command with such semantics would be less than useless. After "mmfsctl suspend" is issued, IOs in progress are allowed to complete, while new incoming IOs are blocked until "mmfsctl resume" is issued. After all IOs already in progress complete, all data and metadata buffers are flushed to disk. Note that this does not mean that caches are cleared. The basic idea of this operation is to provide a self-consistent disk image that would be a good target for a flash copy, i.e. exactly what you're looking for.
Re: how to clear gpfs cache manually?2013-01-18T16:03:29Z in response to SystemAdminHello
Thanks for all your answers.
Concerning the mmfsctl suspend command, as I understand, no IO are losed, while in this suspend mode.
But from the application point of view, what are the impacts ? can we see the application blocked or can we have an IO error, for example, especially if the suspend time is long ?
dlmcnabb 120000P4JT1012 Posts