Measuring the Performance of Your Cloud
As our customers continue their aggressive migration to shared
resource environments in the form of Private Clouds, they are asking
questions about measuring performance.
we took a first step at characterizing a
performance benchmark for Private Clouds. The “Twitter demo”, from
IMPACT, that Andrew Spyker created and Jason McGee walked
one such activity. If you click on the video accompanying this article,
you can watch Jason’s demonstration of the "Twitter demo" and the
IBM Workload Deployer, from IMPACT.
Since IMPACT, we've continued the effort our effort of
benchmarking performance of a private, perhaps even
public, cloud. This article represents our first outline of
characterizing cloud performance.
We decided to start with the 10
Attributes that I described in my
previous blog article on this subject. Now, some of these attributes
don’t necessarily have direct attribution to performance (e.g.,
Standards Based), hence we focused on the ones that did. We ended up
looking at the following six attributes; Time to Deploy, Density,
Scale, Resiliency (including
Security/Isolation) and Runtime
Performance. We also added a final attribute targeted solely
providers, which is Time to Genesis.
Cloud performance is in the eye of the beholder. If you are a
cloud consumer, you simply expect your “service level agreement” to be
met. This includes overall performance, elasticity, security and
resiliency. The tricky job goes to the cloud provider. The provider has
to make the cloud’s resource appear: instantaneous,
infinitely abundant. Hence, this article is predominantly
performance from the cloud providers prospective.
In the IBM
world, virtualization and automation
technological keys to the allusion of "instant, always and infinite".
Hence, many of these areas are
focused on the performance and impact of virtualization, and the
automated provisioning system behind it. The ability to express service
level agreements (SLAs) against your virtual environment is another key
ingredient to measuring performance, allowing the cloud to service the
most important workloads first, and degrade others gracefully.
Here is a quick introduction to these 6 areas of cloud
performance. For each area, I try to provide a simple definition, give
some insight as to why the area is important, and provide high-level
thoughts on how one might set up experiments to quantify performance. I
am boldly assuming that the cloud is mostly used to run enterprise
applications, so the examples follow the behavior of this workload
style. I am hoping in the future to have a more scholarly
perspective, but until then, here is a the high-level overview…
As the pervasiveness of cloud computing continues, measuring
performance of your cloud will be less of an art, and more like a
science (like measuring the performance of web application is today).