Joe Jakubowski, Virtualization Performance Lead Engineer, and Nancy Reaves, Offering and Alliance Manager, recently gave an excellent talk on understanding virtualization benchmarks. Here is what you need to know:
As we know, benchmarks are only one of many ways to evaluate systems. Performance is one component - don't forget to consider reliability features, memory capacity/scalability, I/O, and manageability when selecting a virtualization solution. Virtualization performance (and performance in general) is not just a function of transaction throughput. It is also important to analyze response time. When comparing virtualization benchmark results, it is important to compare like builds and like configurations to make an apples-to-apples comparison. Often a gap in results can be attributed to the hypervisor build. The Build Number can be found in the benchmark disclosure report. SPECvirt_sc2010 is the industry-standard single-host virtualization benchmark for all systems, including x86 servers. This benchmark has many enhancements including: response time criteria applied to throughput metrics memory scalability - the Web server virtual machine will scale with increased virtual memory capacity. application burstiness - the application server is varying the workload and hardware requirements over time – representative of actual usage application/OS level tuning permitted optional results categories that include energy consumption industry developed IBM posted the very first SPECvirt result and is #1 for this benchmark.(1)
(1)SPECvirt_sc2010 result for the IBM System x® 3690 X5 server of SPECvirt_sc2010 1369 @ 84 VMs. Intel® Xeon® Processor X7560 (2.26GHz with 24MB L3 cache per processor—2 chips/16 cores/8 cores per chip), 512GB of memory, one-hundred fourteen (114) 73GB disk drives, Red Hat Enterprise Linux® 5.5, and Kernel-based Virtual Machine (KVM) hypervisor. www.spec.org. Results current as of 11/17/10.
SPEC, SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPECjEnterprise, SPECjvm, SPECvirt, SPECompM, SPECompL, SPECsfs, SPECpower, SPEC MPI and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation (SPEC).
The postings on this site solely reflect the personal views of the author and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.
technorati tags: IBM, systems, performance, virtualization, SPECvirt,burstiness,VMmark,hardware,software,ESX,benchmark,VMware,KVM,hypervisor,Jakubowski,Reaves,scalability,benchmarking