Topic
1 reply Latest Post - ‏2013-01-15T17:04:52Z by laff
SystemAdmin
SystemAdmin
2092 Posts
ACCEPTED ANSWER

Pinned topic how does GPFS pagepool store a big file?

‏2013-01-15T01:47:24Z |
now i have a big file which size is 20g.
i need to read the big file content which start from 50000 and length is 512 byte.
so , i just read a small part of the big file.

i want to know how does GPFS pagepool store the cache? is it like linux os page cache mechanism?

or how do i can view the content of pagepool?
Updated on 2013-01-15T17:04:52Z at 2013-01-15T17:04:52Z by laff
  • laff
    laff
    14 Posts
    ACCEPTED ANSWER

    Re: how does GPFS pagepool store a big file?

    ‏2013-01-15T17:04:52Z  in response to SystemAdmin
    So far as I know... in general GPFS caches Inode/Metadata ..
    if the file itself is cached depends on if it is a sequential read... you can tune that by:

    seqDiscardThreshhold

    One parameter that effects how data is cached in the pagepool is SeqDiscardTheshold.  The SeqDiscardTheshold effects what happens when GPFS detects a sequential access pattern.
    When a sequential access pattern is detected GPFS allocates an optimal number of prefetch buffers (given available memory) and does not retain any of the file data in the pagepool.
    This is the highest performing option for the case where a very large is read sequentially. The default for this value is 1MB which means that if you have a file that is sequentially
    read and is greater than 1MB GPFS will not keep the data in cache. There are some instances where large files are reread often by multiple processes, data analytics for example. 
    In some cases you can improve the performance of these applications by increasing seqDiscardThreshhold to be larger than the file you would like to cache.
    This will tell GPFS to attempt to keep as much data in cache as possible for the file.  The value of seqDiscardThreshhold is file size in bytes.