Create Linux and Windows images for OpenStack private clouds

A new method for OpenStack image construction

This article proposes a new way to construct Linux® and Windows® images for private clouds built with the OpenStack cloud operating system. Current image-creation methods for OpenStack environments are cumbersome and time-consuming. The authors present an online, self-service method that makes image construction faster and easier for operators and end users of private clouds.

Cheng Long Liu (liuclbj@cn.ibm.com ), Advisory IT Specialist, IBM

Cheng Long Liu photoCheng Long Liu is an advisory IT specialist in virtualization and cloud computing. He is the key specialist in cloud services and innovation technologies for the IT-as-a-Service team at the IBM China Development Lab, where he deploys VMware vSphere, IBM SmartCloud Provisioning v2.1, and OpenStack for the lab's internal users.



Zhang Hua (zhuadl@cn.ibm.com), Staff Software Engineer, IBM

author photoZhang Hua is a Staff Software Engineer at the IBM China Development Lab. He works on travel and transportation standards related projects, and his experience includes database, Java, SOA, Web 2.0, and .NET development. He is also interested in Web, search engine, and multimedia processing technologies. He received an MS in Computer Science from Tsinghua University and he has eight years of software development experience. You can contact Zhang at zhuadl@cn.ibm.com.



13 January 2014

Also available in Chinese Russian Japanese

The open source OpenStack cloud operating system is a feature-rich and massively scalable platform for all types of cloud computing. Several public cloud services are based on OpenStack, as are private cloud implementations inside many organizations. But OpenStack still lacks certain features for private clouds, especially for development- and testing-oriented environments. Image construction, for example, isn't a simple process. This article proposes a new and improved image-creation method for OpenStack private clouds. We've verified this new method on the QEMU/KVM platform, but it's also applicable, theoretically, to other hypervisor platforms.

Before introducing the new method, we'll outline how images are created in OpenStack currently.

Existing ways to create images in OpenStack

The processes for creating Linux or Windows images in OpenStack consist of multiple time-consuming steps.

Linux-based images

The official OpenStack Virtual Machine Image Guide details seven requirements that must be met for a Linux-based image to be fully functional in an OpenStack cloud (some of which you can fulfill by installing the cloud-init package). The Image Guide advises users to read that lengthy section of the guide before creating their own images to ensure that the images support the OpenStack features they plan to use.

You can create Linux images either manually or by using tools — such as VMBuilder, Oz, or imagefactory — for some specific distributions. Whichever method you use, you need the following items before you can create your own Linux image:

  • OS installation CD/DVDs or ISO image files.
  • A Linux machine with the KVM/QEMU hypervisor enabled. The virt-manager/virt-viewer GUI utilities might be needed for some distributions.
  • cloud-init or equivalent self-written scripts for the OS.
  • Utilities such as guestfish, guestmount, or virt-* tools for modifying the images.

When all the required conditions are met, you can start to create your own Linux images as per these summarized steps:

  1. Create a virtual machine (VM) and install the OS by using virt-manager or virt-install.
  2. Configure the OS to meet your own requirements (for example, by installing middleware that you need), and to meet OpenStack requirements by installing cloud-init or equivalent scripts.
  3. Modify the image by using guestfish, guestmount, or virt-* to fulfill OpenStack requirements.
  4. Upload the new image to the OpenStack image service and verify the image.

Windows-based images

The OpenStack website doesn't yet include a detailed example of creating Windows-based images. But for a Windows-based image that you create to be fully functional, you must at least:

  • Install a VirtIO driver.
  • Enable Remote Desktop Protocol (RDP) and configure it to pass through the firewall.
  • Configure Internet Control Message Protocol (ICMP) to pass through the firewall.
  • Partition the disk and resize the root partition on boot (using cloudbase-init).
  • Process user data and other metadata (using cloudbase-init).
  • Enable the Windows System Preparation (Sysprep) tool to customize the guest OS.

For most private cloud uses cases, the last two requirements in this list are optional. And you can partition the disk and resize the root partition on boot manually or by script. But you must install a VirtIO driver for the Windows image to work in an OpenStack cloud. In addition, you need to have the VirtIO-Win driver package.

After you satisfy the minimum requirements, the process for constructing a Windows image is:

  1. Create a VM with an Internet Drive Electronics (IDE) disk and an AMD PCnet32 or Realtek rt8139 network interface card (NIC).
  2. Install the OS.
  3. Configure the OS to meet your own requirements (for example, by installing middleware that you need), and to meet OpenStack requirements by installing cloudbase-init or equivalent scripts.
  4. Shut down the VM.
  5. Add a small VirtIO disk and a VirtIO NIC.
  6. Start the VM and install the VirtIO drivers for the VirtIO disk and NIC.
  7. Restart the VM, check the OS, then shut down the VM.
  8. Upload the image to the OpenStack image service and verify the image.

Alternatively, you can follow these steps:

  1. Create a VM with:
    • A VirtIO disk
    • A PCnet32 or rt8139 NIC
    • An additional CD-ROM for the VirtIO disk driver for Windows versions newer than Windows Vista or Windows Server 2008, or an additional floppy for the VirtIO disk driver for Windows versions older than Windows server 2003 R2
  2. Install the OS with the required VirtIO disk driver.
  3. Configure the OS to meet your own requirements (for example, by installing middleware that you need), and to meet OpenStack requirements by installing cloudbase-init or running equivalent scripts.
  4. Shut down the VM.
  5. Add a VirtIO NIC.
  6. Start the VM and install the VirtIO drivers for a VirtIO NIC.
  7. Restart the VM, check the OS, and then shut down the VM.
  8. Upload the image to OpenStack image service and verify the image.

Drawbacks

Despite a few benefits (such as the wide availability of open source tools for Linux-based image construction), the current methods for creating images for OpenStack are not easy. Creating a Windows-based image might seem somewhat easier than the Linux image-creation process, because you don't need to modify the image using tools such as guestfish. But currently no automation tools exist for creating Windows-based images with full functionality for OpenStack, so end users or operators must create them manually. If testers or developers from a globalization team need Windows images, those images must vary by language version — and the team might be using dozens of languages. It would be an impossible task for cloud operators to prepare all the Windows images with all requested language versions.

Both Linux and Windows image creation for a private cloud is time-consuming work for end users — even for experienced cloud operators. Moreover, the organization might lack the resources for end users to create images — for example, the additional KVM/QEMU hypervisor that's required for creating Linux images. In that scenario it would be a huge task for cloud operators to create all the images that end users request.

Finally, new images need to be uploaded to the OpenStack image service, a process that can take a long time depending on the network performance between the image source and the OpenStack image service. For the same reason, verifying new images repeatedly can also take a long time.


Proposed new method for OpenStack image creation

If OpenStack enabled users to create images online, it would be much easier for them to create images that fulfill their needs. We propose a new image-creation method in which users create new images online via the OpenStack dashboard provided by the cloud service. With this capability, end users don't need the additional hypervisor and don't need to upload the images to the OpenStack image service by themselves. All they need is the OS installation CD/DVD ISO image files.

Conceptual design

Conceptually, the ideal process for creating a new image for OpenStack is for end users to:

  1. Upload OS-installation CD/DVD ISO image files to the OpenStack image service.
  2. Launch a new instance from the uploaded ISO image.
  3. Install the OS via the Virtual Networking Computing (VNC)/Simple Protocol for Independent Computing Environments (SPICE) console in the OpenStack dashboard.
  4. Do the necessary configuration as their special needs dictate and install required software packages.
  5. Make OpenStack-required modifications manually or run scripts that the service operator provides — for example, scripts for installing cloud-init, fetching the public SSH key, enabling the SSHD remote login/RDP, and so on.
  6. Take a snapshot of the instance.
  7. Run the glance image-update command on the snapshot as needed to convert the snapshot to an image and add other metadata.

Prerequisites

A few conditions must be fulfilled to ensure that the new image-creation method will succeed:

  • A working dashboard or web UI that's available to all end users.
  • The VNC proxy or SPICE proxy working well and available to all end users.
  • A repository of cloud-init or equivalent script tools available to all end users.
  • An ISO image of the VirtIO-win driver available in the OpenStack image service to all end users.

Next, we demonstrate the feasibility of the new method.


Feasibility study

The two most important aspects of our new method are how the ISO image is supported and how the block device is assembled for the instance launched from the ISO image.

Current support for ISO images

ISO images are supported by OpenStack. Launching an instance from an ISO image is also supported. However, installing a guest OS from an ISO image into an instance launched from an ISO image is not well-supported. For the installation to succeed, some stringent conditions must be satisfied:

  • Guest OSs in ISO images must have VirtIO device drivers enabled by default.
  • The flavor of the ephemeral disk must be set, and its size must meet the requirement of the guest OS.
  • The dashboard and the OpenStack novncproxy server must be running well.

If all these conditions are met, you can install a guest OS from an ISO image successfully to the ephemeral disk of an instance that's launched from this ISO image. And, of course, you can work under the guest OS as you can with other instances. But, because of the current instance snapshot mechanism in OpenStack, you can't successfully convert the instance into an instance snapshot or image. The instance snapshot will include only the root disk of the instance. Other block devices, including ephemeral disks and volumes, will be ignored.

Current assembly workflow for instance block devices

Figure 1 shows the assembly workflow for block devices in OpenStack Nova when you boot a KVM/QEMU instance from the ISO image.

Figure 1. Existing workflow for assembling block devices
Existing workflow for assembling block devices

In the workflow in Figure 1:

  1. Nova retrieves the ISO image from Glance and sets it as the root disk of a VM instance with CD-ROM as device type and IDE as bus type.
  2. Nova creates an ephemeral disk and sets it as the second disk of the VM instance, with disk as device type and VirtIO as bus type. But this step is done only if the ephemeral disk size is set in the instance flavor.
  3. The user installs the guest OS from the root disk (CD-ROM of the instance) to the ephemeral disk (the instance's second disk) and configures it step-by-step through VNC.
  4. The user creates a snapshot from this VM instance, and Nova saves the snapshot to the glance service.

This workflow seems workable for creating a new VM image from scratch. But what you get is a copy of the original ISO image. The reason is that only the root disk (the first block device of the instance, actually the CD-ROM of the instance if it launched from the ISO image) is included in the snapshot, and the ephemeral disk is ignored. So, in current OpenStack, you can launch instances from an ISO image, and you can install an OS in the ISO image to the launched instances with an ephemeral disk configured, but you can't take a snapshot of the ephemeral disk with the OS installed. To address this problem, you need to adjust the assembly workflow of the instance's block devices.

Proposed new assembly workflow

You can change the block device assembly to make an ephemeral disk that's set with proper size and is sure to be set as the root disk for instances launched from the ISO image. After your changes, the root disk included in the instance snapshot will be the ephemeral disk with the OS installed — exactly you want.

Figure 2 shows the assembly workflow of block devices when you boot an instance from an ISO image after modifications (explained in this article's Proof of concept section) are made to the libvirt driver.

Figure 2. Modified workflow for assembling block devices
Modified workflow for assembling block devices

Here's how to modify the block device assembly workflow when launching an instance from an ISO image:

  1. Nova creates a VM disk file and sets it as the root disk of the VM instance. The device bus is set to VirtIO by default.
  2. Nova retrieves the ISO image of the guest OS from Glance and sets it as second disk device, which is a CD-ROM.
  3. Nova retrieves an ISO image of the VirtIO driver from Glance and sets it as third disk device, which is another CD-ROM.
  4. The user installs the guest OS from the second disk device (the first CD-ROM) and configures it as needed.
  5. If the VirtIO driver is not included in the guest OS by default, the third disk device (the second CD-ROM) is used to install the VirtIO driver for the guest OS.
  6. The user creates a snapshot of the instance and Nova saves it to the Glance service.

As explained in Current support for ISO images, the instance snapshot includes only the root disk of the instance, regardless of the root disk's type. With the modified assembly workflow, the root disk is a new disk file created by Nova that contains the guest OS installed from the CD-ROM that's the OS image (the second disk of the instance).

Just as we conceived, the result is an instance snapshot of a new instance installed from the OS ISO image — instead of a copy of original ISO image.


Proof of concept

To ensure that the new image-creation method works as we designed it, we made some modifications to the Nova code — mainly to the libvirt driver. See Download to get the relevant code. The python modules that we modified are libvirt/driver.py and libvirt/blockinfo.py. Our comments in those files identify the class methods and instance methods that we modified.

The environment that we used for our proof of concept consisted of:

Hardware:

  • 2U rack server
  • 2x4 core Xeon Processor
  • 12x8GB RAM
  • 4x900GB SAS hard disk with RAID10 configured
  • 4x1Gps Ethernet card

Software:

  • Red Hat Enterprise Linux 6 update 4 as hypervisor
  • The RDO Grizzly release

We tested the modified code on the RDO Grizzly all-in-one environment, on the multinode installation environment of RDO Grizzly, and on the official OpenStack Grizzly release.


Testing and results

This section describes the simple testing process and some testing examples that we ran with the modified Nova to create VM images with our new method.

Test process

The test process is:

  1. Create OS ISO image to Glance.
  2. Check the existing flavor to make sure the root disk size is the size that you want. Or create a new flavor if there is no applicable one.
  3. Launch an instance from this OS ISO image with the applicable flavor.
  4. After the instance launches, follow the installation steps on the screen to complete the installation of the OS via the VNC console provided by the dashboard.
  5. Install applications as you need and configure the OS as OpenStack requires, such as installing cloud-init or equivalent scripts, enabling SSHD remote login/RDP service, and so on.
  6. Create an instance snapshot of this new installed instance.
  7. Update the snapshot information to change the image type to image by running glance image-update, or from the dashboard if it provides such a function.

Test results

After modifications to the libvirt driver are made, the block devices of instances launched from an ISO image are as shown in Listing 1.

Listing 1. Block device mapping of instances launched from an ISO image
<disk type='file' device='disk'>
 <driver name='qemu' type='qcow2' cache='none'/>
 <source file='/var/lib/nova/instances/290124e3-a267-4223-bd69-661fac2035eb/disk.newos'/>
 <target dev='vda' bus='virtio'/>
 <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</disk>
<disk type='file' device='cdrom'>
 <driver name='qemu' type='qcow2' cache='none'/>
 <source file='/var/lib/nova/instances/290124e3-a267-4223-bd69-661fac2035eb/disk'/>
 <target dev='hda' bus='ide'/>
 <readonly/>
 <address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<disk type='file' device='cdrom'>
 <driver name='qemu' type='qcow2' cache='none'/>
 <source file='/var/lib/nova/instances/290124e3-a267-4223-bd69-661fac2035eb/disk.virtio'/>
 <target dev='hdb' bus='ide'/>
 <readonly/>
 <address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>

Table 1 shows test results for several mainstream OSs.

Table 1. Test results
Guest OSResults
Windows XPFailed*
Windows Server 2003 R2Failed*
Windows 7Success
Windows 8Success
Windows Server 2012Success
RHEL5.9Success
RHEL6.4Success
SLES10 sp4Success
SLES11 sp3Success
*Note: Tests of Windows XP and Windows Server 2003 R2 failed because no floppy disk was found for VirtIO drivers. The root cause is that Windows versions older than Windows Server 2003 R2 only support loading additional disk drivers from floppy disk. In this article, we only add an additional CD-ROM for the VirtIO driver, as shown in Figure 2, but it's possible to add an extra floppy driver into the block devices assembly workflow for older versions of Windows.

Conclusion

The advantages of our new image-creation method are:

  • It's easy to create new images for OpenStack.
  • It's easy to verify the newly created images.
  • A self-service mechanism is available to all end users.

The disadvantages are:

  • Images might not have the full functionality supported — especially, partitioning the disk and resizing the root partition on boot.
  • Windows versions older than Windows Server 2003 R2 are not supported. (But they could be supported if we made the assembly workflow much more complex by supporting floppy drivers for old Windows versions.)
  • Old Linux versions that lack VirtIO device driver support are not supported.

So far, most OpenStack-based public IaaS cloud services provide instances with a fixed root disk size in the base image and — via volume services — additional disk space for the instances. In private clouds, most of the requirements for instances focus on the middleware installed, the versions/editions of guest OSs, instance flavors, and the like. As in public cloud services, disk-size requirements can be fulfilled by providing an fixed average root disk size and sufficient volumes. So, the capability to partition disks and resize the root partition on boot is not a necessity for most private clouds.

OpenStack has grown to be a popular global platform for open source cloud operating systems. It enables cloud solutions of various kinds to be simple to implement, massively scalable, and feature-rich. The work we've presented in this article proves that it's possible to implement new features based on the OpenStack platform — and that OpenStack is an open and flexible framework, not merely a software product.


Download

DescriptionNameSize
Sample codesample_code.zip39KB

Resources

Learn

Get products and technologies

  • Download IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.

Discuss

  • Get involved in the developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Cloud computing on developerWorks


  • Bluemix Developers Community

    Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

  • Cloud digest

    Complete cloud software, infrastructure, and platform knowledge.

  • DevOps Services

    Software development in the cloud. Register today to create a project.

  • Try SoftLayer Cloud

    Deploy public cloud instances in as few as 5 minutes. Try the SoftLayer public cloud instance for one month.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Cloud computing, Open source, Linux
ArticleID=959664
ArticleTitle=Create Linux and Windows images for OpenStack private clouds
publish-date=01132014