Conclusion

With direct I/O, memory consumption is much smaller than in the page cache case.

This is also apparent from the working set sizes reported by z/VM® for the virtual machine (not shown in the diagram). With direct I/O, the resulting lower memory consumption (in terms of memory pages written to per second) results in much shorter relocation times.

The preset throughput does not have a significant impact on the relocation and quiesce times except for the case where no preset fio throughput is set (maximum rate), in which case the quiesce times are about twice as high as with preset throughput.

The fact that the maximum throughputs are almost identical for the page cache and for the direct I/O case results from data constantly being written in random order by the fio processes, using the total of 16GiB file space and the duration of the test (10 minutes).

Consider that writing with a data rate of about 700MiB/sec with no cache hits floods a 4GB cache in about 6 seconds. Latest at that point in time all that data lastly must be written to disk, regardless of whether it is temporarily cached in the page cache, or not. However, the slightly higher maximum data rate in the page cache case indicates that in this case the I/O operations might be a little bit more efficient. For example, I/O operations could be combined and regrouped before data is actually written, or even be avoided at all due to cache hits.

Another aspect is that the I/O subsystem on System z® configured for the test system is able to process high throughputs in both cases, such that in the direct case no significant device and control unit wait times result.