Conclusion

The amount of pages to be transferred in the initial pass depends on the number of pages ever touched by the virtual system before the relocation process started.

In other words, this metric is influenced by the history of activities performed by the virtual system before the relocation process started.

• For the Java™ workload, the amount of memory of 887 k pages (3543 MiB) transferred in the first pass roughly relates to the configuration of the Java heap size (3200 MiB) and space allocated by the Linux® operating system.
• For the page cache file system I/O workload, the high amount of 1019 k pages of memory transferred in the first pass of the relocation process indicates that apparently the virtual memory of that guest was fully utilized for the page cache at that point in time.
  Note: The amount of 1019 k pages (4076 MiB) covers almost all of the guest virtual storage size of 4096 MiB.
• The direct I/O file system I/O workload, the low amount of 74 k pages (296 MiB) of memory transferred in the first pass relates to the set of buffers used by fio for processing the direct I/O operations.
• For the transactional database workload, the amount of 721 k pages (2884 MIB) transferred during the first pass relates to the memory target configured for the Oracle Database server (1616 MiB) plus additional memory used by the database and by the operating system.

The ratio of the average amount of pages transferred in intermediate passes against the number of pages transferred in the first pass gives an indication of the memory write rate and/or memory access distribution of the workload during the relocation process:

• For the Java workload, this ratio is 86%. This indicates a high memory write rate and/or memory access distribution. High memory write rates and a widespread memory access distribution are typical for Java workloads where frequently objects are created and released on the heap, and a garbage collection process periodically recovers the memory from the released objects. As can also be seen from the comparatively high relocation and quiesce times shown in Figure 1, high memory write rates along with a widespread memory access pattern are detrimental to the relocation process.
• For the page cache I/O file system I/O workload, the ratio is 38%. This indicates a moderate memory write rate and/or memory access distribution. Both can be attributed to the ongoing use of the page cache where data written by the fio user processes is temporarily stored before being flushed to respective I/O devices by the Linux guest operating system.
• For the direct I/O file system I/O workload, the ratio is 5.7%. This indicates a low memory write rate and/or a close memory access distribution. This results from the fio processes working only on their I/O buffer and, once prepared, writing these buffers out to the disk storage before using them again for the next write operation. In other words, the memory access distribution in this case is rather limited to the set of I/O buffers. Within the set of observed workloads, this workload has the lowest memory requirements.
• For the transactional database workload, the ratio is 6.7%. This indicates a low memory write rate and/or close memory access distribution. Again, it can be concluded that this is attributed to the database managing its data access through a set of buffers that are frequently reused, keeping the memory access distribution mostly within these buffers and retaining the data as long as possible.

The number of memory passes depends on the memory access behavior performed by the workload during the relocation process:

• For the Java workload it was observed that eight memory passes were used (one first pass, five intermediate passes and two final passes).

https://www.vm.ibm.com/perf/reports/zvm/zvmperf.pdf

• For the page cache file system I/O workloads, higher numbers of memory passes are typically observed . While not shown in Figure 1 , particularly in cases where a preset fio throughput was established (refer to Figure 1), the number of memory passes was between twelve and sixteen. Apparently in these cases the reduced memory modification rate per second lets z/VM perform additional memory passes, successfully minimizing the quiesce time.
• For the direct I/O file system I/O workload, nine memory passes were also observed. Apparently the amount of data transferred during the intermediate passes does not relate to the number of intermediate passes. In other words, nine passes are still seen in this case, but the amount of data transferred per pass is down by a factor of 100 as compared to the page cache fio case. Of course, the duration of each pass is much lesser than in the page cache file system workload case.
• For the transactional database workload, 15 memory passes were observed. Apparently, the transactional database workload nicely fits with the assumptions made by z/VM optimization approaches for keeping the quiesce time low by extending the number of memory passes.

Figure 1 presents the relocation times for the same set of workloads as those shown in Figure 1.

The amount of pages transferred in the final two passes (Figure 1), and the quiesce times (Figure 1) depend on both the size and the activity of the virtual system. However, the quiesce times also seem to have a base component:

• For the Java workload, 594 k pages were observed as being transferred during a quiesce time of 2.78 seconds. These are the highest numbers in our set of workloads, indicating a high degree of memory usage during the progress of the relocation process.
• For the page cache file system I/O workload, 336 k pages were observed as being transferred during a quiesce time of 1.86 seconds. As shown previously (see Figure 1), reducing the fio throughput, results in lower quiesce times, because in the page cache case a lower throughput implies a lower memory modification rate.
• For the direct I/O file system I/O workload, 3.1 k pages were observed as being transferred during a quiesce time of 1.49 s. Surprisingly, the quiesce time stays in the same region as that for the other workloads. Apparently, when only comparatively few pages are to be transferred, activities other than the actual data transfer become more significant for the quiesce time. Furthermore, particularly for a disk I/O benchmark, time is needed until all outstanding disk I/O requests are terminated, also the activation of the devices on the target z/VM.
• For the transactional database workload, 24 k pages were observed as being transferred during a quiesce time of 0.78 s. These are very small values for an active 4 GiB virtual system, and are attributed mostly to the large number memory passes and the locality of memory changes due to special buffers in the database. It can be concluded that the transactional database workload on the Oracle Database is well suited for being relocated while database activity continues.