now i have a big file which size is 20g.
i need to read the big file content which start from 50000 and length is 512 byte.
so , i just read a small part of the big file.
i want to know how does GPFS pagepool store the cache? is it like linux os page cache mechanism?
or how do i can view the content of pagepool?
This topic has been locked.
1 reply Latest Post - 2013-01-15T17:04:52Z by laff
Pinned topic how does GPFS pagepool store a big file?
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
Updated on 2013-01-15T17:04:52Z at 2013-01-15T17:04:52Z by laff
laff 100000UC2914 PostsACCEPTED ANSWER
Re: how does GPFS pagepool store a big file?2013-01-15T17:04:52Z in response to SystemAdminSo far as I know... in general GPFS caches Inode/Metadata ..
if the file itself is cached depends on if it is a sequential read... you can tune that by:
One parameter that effects how data is cached in the pagepool is SeqDiscardTheshold. The SeqDiscardTheshold effects what happens when GPFS detects a sequential access pattern.
When a sequential access pattern is detected GPFS allocates an optimal number of prefetch buffers (given available memory) and does not retain any of the file data in the pagepool.
This is the highest performing option for the case where a very large is read sequentially. The default for this value is 1MB which means that if you have a file that is sequentially
read and is greater than 1MB GPFS will not keep the data in cache. There are some instances where large files are reread often by multiple processes, data analytics for example.
In some cases you can improve the performance of these applications by increasing seqDiscardThreshhold to be larger than the file you would like to cache.
This will tell GPFS to attempt to keep as much data in cache as possible for the file. The value of seqDiscardThreshhold is file size in bytes.