With a steady flow of improvements and bug fixes to existing software, the open source world is in constant flux. Staying at the bleeding edge of software upgrades can be a full-time job. One of the trickiest parts about upgrading your software is that you never really know if your applications are going to work after the upgrade is complete. Most software packaging systems offer a rollback feature, but that's often not enough; ideally, you want to play with these new updates to test and try them in an environment where they can't hurt anything.
Like a kid on the playground, you want your own sandbox to play in, where you can make a mess and not worry about picking up.
Virtualization is often used to separate (or "sandbox") applications and systems from the others on the same hardware. Linux supports many different forms of virtualization ranging from hardware emulators to pure hardware virtualization. One recent standout in the growing list of virtualization technologies is Xen, developed at the University of Cambridge. Xen is noteworthy because of its unprecedented performance and security.
Developed as a research project at the University of Cambridge, Xen has gained a lot of momentum in the open source community. Xen is a paravirtualizing VMM (Virtual Machine Monitor), meaning that the operating system is modified in select areas to make calls into the hypervisor, whereas the applications that run on that operating system are unmodified. While other virtualization systems like VMWare demonstrate true virtualization (in which they don't have to modify the running operating systems), they still need to do on-the-fly translation of the machine code, which can affect performance.
Because Xen requires a modified operating system, you cannot just take a current Linux kernel and run it under the Xen hypervisor unless it is ported to the Xen architecture. However, if the current system can use a new Linux kernel that has been ported to the Xen architecture, you can run your existing system without change.
Figure 1. The simple Xen architecture
The source code for Xen is available from the project page (see the Resources section for the link), but if you're already running a Linux distribution, you may be able to get Xen from a package update or installation. These major distributions are packaging Xen:
Check with your distribution -- if it is not in this list, you may have to use the source code directly. Beyond distribution-specific install procedures, everything works the same in Xen on any distribution.
I have a working installation of Fedora Core 3, and to keep this article brief, I'm going to talk about getting the system up and running on a Fedora Core.
Fedora sports a great community of Xen users already, even though the packages have only recently landed in the development channels. Because of the number of people trying out Xen on Fedora, you're probably going to have lots of success finding online help.
To help you get up and running quickly, this section highlights the major points of the Quick Start Guide from the Resources section . As Xen updates are pushed into Fedora, the Quick Start Guide will most likely change over time and hopefully the wiki will remain updated to those changes.
Start with a minimal server installation for the base machine. This base machine is your hypervisor and won't be running any of the server applications. You'll have a chance to install the packages you want on the Xen servers that run atop this machine, so there's no need to install the applications you want to run just yet.
Once you have a system running, you'll need to update it to the latest development version of Fedora, called rawhide. You can do this by going through the elements in the /etc/yum.repos.d/ directory and changing all of them except the fedora-devel.repo to be
enabled=0; change fedora-devel.repo to
Now that your yum repository is set up, you need to update the machine to rawhide so you can start using the latest Xen release.
Listing 1. Update FC3 to rawhide and install the Xen packages
yum update yum install xen kernel-xen0 kernel-xenU
Next, create a dummy filesystem to work with.
Listing 2. Format and set up Xen server filesystems
mkdir -p /xen/base dd if=/dev/zero of=/root/base.img bs=1M count=1 seek=1024 mkfs.ext3 /root/base.img mount -o loop /root/base.img /xen/base
With your Xen server filesystems formatted and mounted, the next step is installing the packages necessary to run something on them. To run a Web server on your Xen servers, you'll need to install the Web-server set of packages. (Warning: This will take a while! Run yum, then go for a walk.)
Listing 3. Installing Xen server packages
yum --installroot=/xen/base -y groupinstall web-server \ --enablerepo=base --disablerepo=development
(You should be walking now.)
You now have all the packages you need installed, but you need to drop in a simple fstab file, which tells the Xen servers that they are going to be getting a root device sda1, which they will use as root. This root device is actually a virtualized device coming from the hypervisor, but your servers won't know that.
Listing 4. Xen server fstab file
/dev/sda1 / ext3 defaults 1 1 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0
Write this file into /xen/base/etc/fstab. Without a normal Fedora install, no fstab was created for you. Future versions of Fedora are expected to include a tool for handling this issue.
Finally, you need to do some hacks to get the system running properly. Xen is currently having issues with
initrd, so you need to create a few device codes for yourself if you want the system to boot. Then, in order to keep things running smoothly, it's beneficial to move the TLS libraries; at the moment, they interact badly with Xen. Once you've done that, you can unmount this filesystem.
Listing 5. Final hacks to get running
for i in console null zero ; do MAKEDEV -d /xen/base/dev -x $i ; done mv /lib/tls /lib/tls.disabled mv /xen/base/lib/tls /xeb/base/lib/tls.disabled umount /xen/base/
Now that you've painfully set up your guests by hand, you can actually reboot the machine and start using the Xen kernel. You'll probably see a lot more printouts than normal, and you'll also get scary message about the TLS library (the one you moved out of the way in the previous section).
Xen requires configuration files for each server you run. Although you can create a single dynamic configuration file for all of your servers, I recommend creating two static configuration files, one for each. As you can see, the disk label tells the servers that your file image is a sda1 device. The xenU kernel you are using and the memory that the server guest will receive are also specified.
Listing 6. Xen server configuration files
/etc/xen/base kernel ="/boot/vmlinuz-2.6.10-1.1141_FC4xenU" memory = 64 name = "BaseServer" nics = 1 disk = ['file:/root/base.img,sda1,w'] root = "/dev/sda1 ro" /etc/xen/test kernel ="/boot/vmlinuz-2.6.10-1.1141_FC4xenU" memory = 64 name = "TestServer" nics = 1 disk = ['file:/root/test.img,sda1,w'] root = "/dev/sda1 ro"
To get the servers up and running, start up the Xen service with the command
xend start and then create your BaseServer from your base config file with the command
xm create base. With those actions, you'll be taken into your Xen guest and you can watch it boot up. If at anytime you want to escape the guest console, press Ctrl-] and go right back to the hypervisor console.
You've now spent some time editing configurations and getting your system set up to run a couple of Xen servers. After all that hard work, what is your reward?
Now you have two copies of a system running the same kernel, virtualized on the same machine. If your BaseServer represents the stable environment that you'd like to run your Web service on normally, then your TestServer can represent the latest software updates that Fedora rawhide provides. You can keep upgrading your TestServer system at will, trying all the new versions of software updates -- when you feel it's stable enough, you can make the TestServer run as the BaseServer.
When the BaseServer is operating, it's a good time to change the configuration for the apache httpd service you'll be running. You might want to turn off ssl by moving the ssl.conf file out of the /etc/httpd/conf.d/ directory; otherwise, you'll need to generate a certificate for the servers. Also you might need to add an apache user to the system.
The Xen hypervisor has automatically given you a virtual network device to work with. If you can run dhcp on this device, just run
dhclient eth0; you should get an IP address for your BaseServer.
Once you have your configuration setup for the BaseServer, run
poweroff from the BaseServer console to shut down the instance. If you've already logged out of the BaseServer, regain access to it using
xm console BaseServer.
Now copy the base image file by using
cp /root/base.img /root/test.img so you can have a duplicate copy of the same filesystem. When you run
xm create -c test, it runs the same server as the BaseServer, but it's called TestServer. Log into the TestServer and enable rawhide support like you did in the Installing Xen section, and run
Your TestServer instance has all the same configurations as your BaseServer did, but you've upgraded to the latest packages that Fedora rawhide has to offer. Here's where you get to play around to see if your Web site is still working.
I hope these quick instructions and simple example help you get Xen up and running so you can try it out. In the interest of getting you familiar with Xen quickly, I did not cover how to take advantage of things like LVM and snapshots, or how to do a network migration of a Xen server from one machine to another. But now that you've seen some of the advantages that Xen can provide, explore the Resources below to learn some of the more fancy tricks you can do with it.
- Find mailing lists and other channels of support for Xen users at the Xen Project Page.
- "Xen and the Art of Virtualization" is a detailed research paper that describes the Xen hypervisor architecture. This and the other papers on the Xen architecture are invaluable in understanding exactly how Xen works.
- "Xen and the Art of Repeated Research" is a performance comparison of XenoLinux (Linux running in a Xen virtual machine) to native Linux as well as to other virtualization tools on an IBM eServer™ zSeries® mainframe. It repeats an earlier performance analysis for consistency of results.
- See the Xen in Fedora Core Project for project update information.
- To get Xen up and running and to share your experiences, use the wiki page on the Xen Quick Start Guide for Fedora from the fedoraproject site.
- The FUDCon: Fedora User and Developer Conference is open to everyone and recently included a talk on Xen at the first conference.
- Get Xen'ed for these distributions: SUSE Linux, Debian, and Gentoo's bugzilla (experimental e-builds).
- "Virtualization and the On Demand Business" (IBM Redpaper, August 2004) describes how organizations can use virtualization as a technique to gain more business value and greater flexibility from their IT infrastructure.
- "Autonomic features of the IBM Virtualization Engine" (developerWorks, September 2004) introduces the title engine, a suite of systems services and technologies that provides a way to manage resources, servers, storage, and networks as an integrated system rather than as individual components.
- "Architecture for Virtualization with WebSphere Application Server, Version 5" demonstrates how to use the WebSphere® Application Server to build virtualization resources.
- "Advanced Virtualization Features on p5 Servers" (IBM Redbooks Technote, February 2005) demonstrates how to select the best virtualization configuration for your needs on an IBM eServer pSeries® 5 server.
- Developer resources for an on demand world offers a roundup of resources on virtualization.
- Find more resources for Linux developers in the developerWorks Linux zone.
- Get involved in the developerWorks community by participating in
- Browse for books on these and other technical topics.
- Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
- Innovate your next Linux development project with IBM trial software, available for download directly from developerWorks.
Bryan Clark makes monkey noises for Red Hat's Desktop Group. Also a member of many projects on the GNOME Desktop, Bryan tries to be as active as possible in taking over the world's desktops. Bryan and his crew from the Clarkson Open Source Institute published a research paper last year to re-evaluate the Xen performance statistics. He currently lives in Cambridge, Massachusetts, and loves traveling to other places just as interesting. Contact Bryan at email@example.com.