Monitoring memory usage within the cluster caching facility server process

The CF_MEM_SZ cluster caching facility (also known as a CF) memory configuration parameter controls the maximum size that the CF process can grow to. When the CFs starts up, using memory tracking utilities such as svmon, will not show a large change in used memory because the full allocation is not yet committed or backed by real memory. Memory is only committed or backed when it is accessed for the first time. As more databases are activated and database activities take place, more memory is committed in order to keep track of the activities until the maximum CF memory size is reached.

The memory for certain structures persist until a database is dropped or the CF server is stopped therefore the memory might persist as shown in svmon output.

For example, the following Db2® pureScale® instance has the CF configured to use around 132GB.
Memory size (4KB)                         (CF_MEM_SZ) = AUTOMATIC(33874432)
The top command shows the memory usage of the CF at only 5% after startup.
 PID USER      PR  NI    VIRT    RES    SHR S   %CPU  %MEM     TIME+ COMMAND                                                                                                                                                                
 6168 dtw       20   0  0.132t 9.498g 9.223g S 2988.2 5.028 178819:52 ca-server                                                                                                                                                              
 3070 root      20   0  841600   3128   2512 S  5.882 0.002   7:13.75 nscd                                                                                                                                                                   
 4135 root      20   0  433752  45292  19032 S  5.882 0.023   1544:02 BESClient                                                                                                                                                              
 6652 dtw       20   0   15644   3020   2272 R  5.882 0.002   0:00.02 top
The db2pd -cfinfo command can show more detail about the CF memory usage. Here is a sample output:
CF Server (128) Information
    Hostname                                = coralm215.torolab.ibm.com
    Management Port                         = 27727
    Role                                    = Primary
    CF Host Information
       Virtual Memory Used                  = 18793431040
       Virtual Memory Available             = 188314128384
       Real Memory Used                     = 18793431040
       Real Memory Available                = 184012312576
       Swap File Used                       = 0
       Swap File Available                  = 4301815808
       CPU Usage                            = 0
       CPU Activity                         = 97
    Total structure memory (4KB)            = 34626048
    Free structure memory (4KB)             = 2724096
    Memory frame Size (4KB)                 = 256
    Configured Size (4KB)                   = 34626048
    Current Worker Queue Size               = 0
    Minimum Worker Queue Size               = 0
    Maximum Worker Queue Size               = 1
    Average Worker Queue Size               = 0
The db2pd -cfinfo -db < dbname > command can be used to see the current memory allocation of each database structure in the CF. As shown in the following example, the current size of the GBP structure is much smaller than the configured size. This size will grow as the structure is used by database activity.
    Group Bufferpool Information
        Structure Name                      = db2.dtw.DTW.gbp
        SID                                 = 316
        Current Size (4KB)                  = 1260227
        Target Size (4KB)                   = 30700032
        Configured Size (4KB)               = 30700032
        CF Driven Castout                   = 0


    Shared Communication Area Information
        Structure Name                      = db2.dtw.DTW.sca
        SID                                 = 314
        Current Size (4KB)                  = 5198
        Target Size (4KB)                   = 58624
        Configured Size (4KB)               = 58624


    List Structure / Smart Array Information
        Structure Name                      = db2.dtw.DTW.list
        SID                                 = 337
        Current Size (4KB)                  = 14339
        Target Size (4KB)                   = 141568
        Configured Size (4KB)               = 141568


    Lock Structure Information
        Structure Name                      = db2.dtw.DTW.lock
        SID                                 = 318
        Current Size (4KB)                  = 249742
        Target Size (4KB)                   = 1000192
        Configured Size (4KB)               = 1000192