Virtual appliances and the Open Virtualization Format
A format and toolkit for cross-hypervisor appliances
Virtual appliances are an evolution of software delivery enabled through virtualization. They are fueling the delivery of efficient and optimized applications and also the quickly growing wave of cloud computing. This article explores the ideas behind virtualization, virtual appliances, their implications for operating systems, and coming standards that will simplify the deployment of virtual appliance software.
The key to virtual appliances
Let's start with a quick discussion of the key behind virtual appliances: the virtualization platform. Although operating systems represent a virtualization platform for applications (sharing access to the processor and resources), hypervisors do the same thing for operating systems. Hypervisors are important, because they allow you to run multiple, different operating systems on the same system. From the perspective of a desktop user, this ability is important, because it allows them to run Windows® and Linux® concurrently. But from the perspective of a server, it permits hosting multiple operating systems running a variety of server applications concurrently (such as Web servers, e-mail applications, databases, and other useful services). Further, each operating system is isolated from one another, so a fault in one does not affect any other.
For server applications, virtualization is crucially important. A recent U.S. Environmental Protection Agency study found that typical servers were used to only 5% of their capacity. That's a lot of energy, space, and cost associated with a system. Server virtualization solves this problem by creating a virtualized server on which many operating systems (and their applications) can run. In this way, 10 or more physical servers can be reduced to 10 or more virtual servers running on a single physical server (see Figure 1).
Figure 1. Benefits of server virtualization
In Figure 1, you can see that each software stack (operating system and applications) is an isolated virtual machine (VM). The failure of one VM does not mean that all other VMs fail. But what's most important in this picture is at the top of each stack. The purpose behind each software stack is the application level: The applications are the real value in the software stack. In this sense, virtual appliances are simply containers for applications. Applications have their own dependencies (such as an operating system or libraries), but the applications are the key.
Anatomy of virtual appliances
So, a virtual appliance is simply a container for applications and all their dependencies (operating system, third-party libraries, configuration, and so on). Let's explore the virtual appliance—as well as its alternative—in a bit more depth.
The virtual appliance container is a VM image (VMI). A VMI is a file that contains the VM's hard disk image (including the kernel, boot partition, and root file system). The file is typically in a compressed format—specifically, the file grows and shrinks with the contents of the VMI and is not defined by the capacity of the VM's hard disk. The format of the VMI is important for compatibility reasons, and I'll return to this topic shortly.
You can think of a virtual appliance, then, as a software stack that incorporates the target applications, libraries, services, configuration, relevant data, and operating system. Further, the virtual appliance, in addition to being a set of combined elements, is configured and tuned for the specific application (see Figure 2).
Figure 2. Anatomy of a virtual appliance
But at the end of the day, the virtual appliance is nothing more than a large file that a hypervisor can use to create a VM instance. As a file, the appliance has some useful properties—for example, you can duplicate it (for backup purposes) as well as move it between hypervisors (called VM migration).
The advantage of the virtual appliance is that the focus is on the application. The VMI incorporates everything necessary to run that application in a virtualized environment. It is configured and tuned for that application, simplifying its deployment. Now, consider the alternative: That same application could be distributed outside a VMI and run on an arbitrary operating system, requiring that you build the application (and all dependent libraries and services), then configure and tune the application and environment. The application may have specific operating system requirements, as well. This process is common, and its complexity depends on the application itself. But the process is time-consuming and prone to error.
The fundamental difference comes back to a focus on the application. For example, an application installation can entail installation of a specific operating system (or the installation of a specific file system). The installation can also rely on installation of dependent services (such as a database) and libraries (in addition to satisfying their dependencies). Finally, many of these elements may require configuration and tuning. Once the application is deployed to a particular system, it is tied to that system.
The VMI relies on a compatible hypervisor and potentially some configuration of the hypervisor's device emulation, but the focus remains on the application. There's more value to a virtual appliance than just simpler configuration, however. Now, let's look at one of the key aspects of the virtual appliance.
Implications of virtual appliances
When you start to look at virtual appliances from the perspective of the application, you can then begin to tailor and tune the entire environment around that application. Consider the operating system. For a given application, certain operating system services may not be necessary, although other services may be important to the target application's performance. The operating system in this context is referred to as a Just Enough Operating System, or JeOS (pronounced juice). The term was first coined by VMware and represents the ideal construct for the virtual appliance.
Although VMware coined the term, however, the practice has been around for a considerable time. Linux-based embedded systems typically employ a smaller footprint kernel and application stack that is better for the constrained environment than an embedded setting provided. However, VMware's usage goes beyond simply minimizing services to support a constrained environment. Hard disk space and memory are typically plentiful in server environments, but there's actually much more at play. Fewer services and drivers mean less software to maintain on a given system. From a management perspective, this means fewer patches to keep the operating system up to date (with less bandwidth and storage needed).
From the perspective of reliability, JeOS offers some advantages. By excluding software that isn't necessary for a given system, you reduce the opportunities for failure and increase reliability, which also improves the security of the software stack by minimizing the possible exploits (fewer services mean fewer paths to attack a system).
Linux (a monolithic kernel) provides the means to tailor the operating system dynamically through modules, but larger architectural minimizations are only available through kernel compilation. In contrast, microkernel operating systems provide the means to discard functionality dynamically, allowing a much more scalable environment. Microkernels do this by creating a kernel (which includes the basic services like memory management and scheduling) and pushing all other functionality outside the kernel. In addition to providing a more scalable environment, the microkernel can offer reliability advantages. The less code that runs in privileged space, the less likely it is that an operating system will fail.
Building virtual appliances
To minimize the Linux kernel, you can go a couple of distinct routes. The first is manual configuration of a kernel through the kernel build process. Although this may be the most precise way to minimize a kernel (for your particular application), there is a significantly simpler method. Most of the major Linux distributions provide JeOS variants. These operating system derivatives are configured specifically for virtual appliances running on a hypervisor (see Related topics for links to more information).
The Ubuntu JeOS is a Ubuntu derivative optimized to run on VMware ESXi or VMware Server or the Kernel-based Virtual Machine (KVM) Linux hypervisor. It includes no graphical environment and runs on the Intel® or AMD x86 architecture with a minimum memory footprint of less than 380MB. Ubuntu JeOS also includes the ubuntu-vm-builder utility, which you can use to build virtual appliances (within minutes) based on a user-defined specification.
SUSE Linux also provides a JeOS environment called the SUSE Appliance Program. This environment is similar to Ubuntu's distribution but with a few notable differences. The SUSE variant works with the SUSE Linux kernel and provides a couple of different tools for building virtual appliances, depending on your specific need. SUSE provides Kiwi, a command line-based tool that lets you create an appliance specification through XML. With Kiwi you can build a virtual appliance in a variety of different formats, including LiveCD, bootable USB image, or OVF. There's also an appliance-creation tool for manual construction of virtual appliances. Finally, SUSE Studio provides an easy-to-use Web-based appliance-creation tool that offers point-and-click construction.
Two other offerings—Oracle Enterprise Linux JeOS and OpenSolaris JeOS project—provide similar capabilities for building small-footprint virtual appliances. But one of the oldest commercial solutions, rBuilder from rPath, can be used to build virtual appliances online. Using a Web-based front end, rBuilder allows you to build public virtual appliances using rPath's Linux distribution or with CenOS or Ubuntu.
Finally, the OVF Toolkit is a set of Eclipse plug-ins and a Java™ API that you can use to build virtual appliance packages in the OVF. The Related topics section provides a link to a great tutorial covering the use of this tool.
Virtual appliances and the OVF
Virtual appliances are useful only as long as they are portable. The ability to take a virtual appliance and run it on VMware, as you could on Xen or KVM, means more opportunities for the appliance. This is where the OVF (previously called the Open Virtual Machine Format) comes in. The OVF is a specification for a portable (meaning hypervisor-neutral) distribution method for VMs (or in this case, virtual appliances). With OVF, you can securely package and distribute virtual appliances in an efficient way. OVF can even represent multiple VMs in a single package. The OVF is extensible, permitting extensions as virtualization moves forward.
The OVF is an XML format that describes all the various aspects of a VM (in disk format). This includes a description of the virtual disks that make up the VM; the network configuration for the VM; processor and memory resources required for the VM; and a variety of metadata describing the virtual appliance creator, the purpose of the VM, and an operating system description. In the Related topics section, you can find a link to the VMware OVF white paper, which includes a complete example of an OVF file.
Additionally, tools exist that provide conversion of older format VMs into
the OVF. For example, you can use
convert older-style .vmx VM files into the OVF. You can also find tools
that convert from VMware and QEMU to VirtualBox as well as from VMware to
Xen and Parallels.
The future of virtual appliances
Virtual appliances are a simple path to complex applications by combining preconfigured applications and a (just enough) operating system into a single image ready to run on a hypervisor. You can find virtual appliances that cover almost any application, from firewalls and security appliances to servers, backup appliances, and storage applications (including replication and archival).
Virtual appliances not only simplify the deployment of applications for individual users but also power next-generation cloud computing architectures. Rather than the considerable time required to build a specialized distribution with applications, most cloud computing infrastructures provide ready-to-deploy virtual appliances to satisfy any need. And because a virtual appliance is simply a file with a wrapper (the XML description), it's easy to replicate and distribute such appliances.
The OVF specification is a joint development by a number of companies involved in both virtualization and hardware development, including IBM, Microsoft, Hewlett-Packard, Dell, VMware, and XenSource. When big players like these can agree on a specification like OVF, you know it's going places. The growing development around virtual appliances will ultimately drive their continued growth, and new specifications like the OVF will certainly help.
- "Virtual Linux" (developerWorks, December 2006) provides more information on the various Linux-based virtualization schemes. This article explores full virtualization, para-virtualization, and operating system virtualization.
- The term JeOS (Just enough Operating System) was first coined by VMware's Srinivas Krishnamurti. JeOS refers to an operating system that has been scaled back to include only those services that the target application requires.
- Ubuntu Server Edition JeOS is a variant of tUbuntu's Linux distribution that is designed specifically for virtual appliances. Ubuntu also provides a nice page dedicated to building virtual appliances with JeOS, including operating system preparation and installation of necessary applications.
- The SUSE Appliance Program is Novell's JeOS distribution for the SUSE Linux distribution.
- rBuilder available from rPath is an interesting commercial virtual appliance builder. This online tool allows you to build virtual appliances using CentOS, Ubuntu, or rPath's own Linux distribution.
- In "Build virtual appliances using the OVF Toolkit" (developerWorks, June 2009), learn how easy it is to build virtual appliances using the OVF Toolkit as an Eclipse plug-in.
- VMware's list of virtual appliances developed with VMware Studio 2.0 Beta includes a wide range of virtual appliance examples from security, storage, administration, content management, and collaboration.
- In "Linux virtualization and device passthrough" (developerWorks, October 2009), learn more about device emulation. Device emulation is an important part of the hypervisor, bridging the gap between virtual devices in the VM to physical devices below the hypervisor.
- In "Reduce your Linux memory footprint" (developerWorks, January 2007), you'll learn how to tweak your operating system for smaller footprint and greater performance. Although more manual than virtual appliance tools, this method provides the greatest control over building JeOS.
- This OVF white paper provides a great introduction to the OVF and its purpose. Also in this white paper is an example OVF file illustrating a virtual appliance. You can find a variety of tools to support VM creation using OVF—for example, under development is the Open-OVF available at SourceForge.
- The ovftool allows you to easily migrate VMs in certain formats to OVF as well as back from OVF to other formats.