Summary

In this study we investigated the performance of a fixed set of workloads running in z/VM® virtual systems while z/VM real memory and processors were progressively constrained. Scaling down the amount of z/VM real memory and the number of z/VM real CPUs caused an increasing ratio between virtual resources and real resources. The real resource shortages had implications on the performance of the workloads, with applications either waiting for getting scheduled on a CPU or for virtual memory being paged in from z/VM paging devices. The focus of the study was on the new z/VM 6.3 release that introduced major improvements in memory and processor management. Another aspect analyzed was the use of EDEV-SCSI devices for z/VM paging.

From an initial investigation performed in an oversized z/VM environment (25 real CPUs and 512 GiB real memory) it ensued that the workload set required about 21 real CPUs and 133 GiB real memory.

In subsequent iterations:

• the number of real CPUs was scaled down from 20 to 5,
• the amount of real memory was scaled down from 512 GiB to 64 GiB.

When scaling down the number of real CPUs, the overall workload throughput scaled about linearly, albeit the individual workloads scaled differently. This highlights an important aspect of the test setup: There was fierce competition for resources by the workload mix. Depending on the requirements of the individual workloads, it might happen that the performance of one workload degrades, while another workload continues unaffected or even profits. For example, one workload might wait for memory to be paged in and is thus unable to use CPU, while a second workload has low memory requirements and thus is able to make use of CPU not consumed by the first workload.

When scaling down the amount of real memory, the overall workload throughput stayed flat and even increased initially, until real memory was configured to be less than required as initially determined. Beyond that, the overall throughput decreased more or less sharply, depending on the z/VM environment. Here the performance of the z/VM 6.3 environment was superior to that of the z/VM 6.2 environment, and the use of EDEV-SCSI paging devices was superior to that of ECKD™ paging devices. Using the z/VM 6.3 environment and EDEV-SCSI paging devices, it was possible to scale z/VM real memory down to 64 GiB, while all workloads continued to operate in a stable manner, albeit with degraded performance. Note that 64 GiB real memory is less than half of the real memory that was required to operate the workload set without real memory constraints.

Another focus was on workload specific behavior:

• Database BI workload within a large virtual system

  The database BI workload was less impacted by z/VM real memory constraints. When in addition z/VM real CPUs were also constrained, the workload mostly was less affected, and in some cases even could increase its throughput. This is attributed to the sophisticated mechanisms used for caching and for asynchronously scheduling work that are commonly applied by modern databases. A noteworthy characteristic of the database BI workload was that it was operated in a virtual system exhibiting a very large virtual memory (300 GiB), whereof the workload on average actually used about 80 GiB. This is still about five times larger than the next largest virtual system used in the workload set.

• Transactional WAS workload

  The transactional WAS workload was executed in two differently sized virtual machine sets, imposing two workload levels – medium and high. Both levels behaved very similar. Once the z/VM real memory overcommitment exceeded a certain level a distinct impact was observed. The impact was less significant than that observed for the (pure) Java™ workload. A possible explanation for this behavior is that the transactional WAS workload used the gencon garbage collection policy that might have helped limiting the spread of memory accesses.

• Java workload

  The Java workload was also executed at two levels - medium and high - that behaved very similar. By trend, the z/VM workload also behaved similar to the transactional WAS workload. However, the setback of the throughput already occurred at higher z/VM real memory settings. A possible explanation is that this workload used the optthruput garbage collection policy that might have caused widespread memory accesses.

• File system I/O workload

  The file system I/O workload was executed in two variants, direct I/O and page cached I/O, and each variant was executed at two levels. The use of page-cached I/O versus direct I/O caused very different memory usage patterns. The page-cached variant used high amount of virtual memory (up to the complete virtual memory defined for its virtual system), causing relatively high page-read rates when z/VM real memory was constrained. Opposed to that the direct I/O variant had a very small memory footprint, but with intense memory usage (that is, high memory access rates). This caused most of the used virtual memory to be kept resident by z/VM, such that no page-read activity from the z/VM paging space occurred.

• Network workload

  The network workload was executed at two levels that behaved very similar. The network workload also behaved much like the direct I/O variant of the file system I/O workload, that is, the memory footprint was small but with intense usage, avoiding z/VM page-read activities for this workload.

Table 1 shows a categorization of various workloads with respect to their memory overcommit tolerance.

Table 1. Categorization of various workloads with respect to their memory overcommitment tolerance

		Remaining resident pages under memory pressure [1]
		low	medium	high
Page read rate	low	Network File system with direct I/O
	medium	DB2® transactional as WebSphere® backend	Web Server Medium level file system I/O with page cache	high level file system with page cache
	high		Database BI WebSphere Application Server	Java
[1] Remaining resident pages under memory pressure classifies the workload with respect to the amount of resident pages that the workload still effects when running in a memory constrained z/VM environment, as opposed to an unconstrained z/VM environment.

The workloads are characterized based on their page read rates and on the amount of virtual memory backed by real memory, when running in a memory constrained z/VM environment.

• The upper left corner represents workloads that are able to tolerate z/VM real memory constraints. Typically such workloads have a small memory footprint, but with intense memory usage, causing low z/VM page-read rates. When z/VM real memory is over-committed while executing a mixed workload set, these workloads are typically only moderately impacted.
• The lower right corner represents workloads that have a large memory footprint, along with widespread use of the memory, causing significant z/VM page-read rates. These applications have a high chance of showing performance impacts already when only small z/VM real memory constraints exist.

It is emphasized that the mixing of workloads turned out to be an essential factor that influences the potential for z/VM real memory overcommitment. A mixture of different kinds of workload enables some workloads to profit from other workloads when these are unable to make use of still available resources (such as for example CPU) while waiting for another essential resource (such as for example memory).