Establish a system to build custom virtual cloud images

Using the IBM Image Construction and Composition Tool

Virtual images allow users to freeze-dry preferred software environments and deliver them to users in a rapid and consistent manner: As a result, companies are turning to virtual images as a means to improve software delivery within their data centers. As they make greater use of virtual images, challenges are emerging — such as determining how much content to put into a single image and how best to construct these images. The authors discuss these challenges and introduce the IBM® Image Construction and Composition Tool (ICCT), freely available for download (see Resources), that addresses many of these challenges and provides a systematic approach to constructing virtual images in the cloud.

Share:

Dustin Amrhein, Technical Evangelist, IBM

Author photoDustin Amrhein joined IBM as a member of the development team for the WebSphere Application Server. While in that position, Dustin worked primarily on web services infrastructure and web services programming models. In addition, Dustin worked on the development of a RESTful services framework for Java runtimes. In his current role Dustin is a WebSphere Client Technical Professional.


developerWorks Professional author
        level

Vanessa Grose, Advisory Software Engineer, IBM

Author photoVanessa began working for IBM in System i Java Virtual Machine development and then joined the Application Integration Middleware (AIM) Customer Programs organization. In her current role, Vanessa provides education and training and gathers customer feedback on products in the AIM portfolio, focusing on IBM Java technology, WebSphere Application Server, and WebSphere CloudBurst Appliance. She recently completed a Master of Science degree in Computer Science from the University of Minnesota.



07 December 2011 (First published 22 March 2011)

Also available in Chinese Japanese Portuguese

If you are in the business of constructing and delivering software environments for your own or others' use, you are likely painfully aware of how hard this can be. With traditional approaches, you have to gather requirements for the environment, install software, configure each component, integrate multiple components, and then cross your fingers and hope everything went right. The process can be timely, error-prone, and quite honestly maddening!

It is unlikely that many of you would consider the tasks of installing and configuring basic software environments as a competitive advantage; rather, you install and configure these environments so that you can do productive work, such as application development. You understand this reasoning — as it happens, many companies have also come to the realization that they need a better way to deliver their software environments over the course of the last several years.

In light of this realization, many companies have embraced the use of virtual images as a means to conquer the traditional challenges of environment construction and delivery. While virtual images may offer less flexibility when compared with traditional installation approaches, they overcome this by enabling you to freeze-dry the installation and, to some degree, the configuration of your environments.

What this means is that you only have to perform those installation and configuration tasks once.

In addition, virtual images allow you to quickly deliver your configured environments. All you need to do is activate a virtual image and you have a machine with the required components already installed.

Due to the obvious benefits, the adoption of virtual images continues to increase; however, this increase in adoption brings new challenges, challenges introduced by the virtual image delivery approach.

A new approach means new challenges

The benefit of being able to stand up configured environments in a matter of minutes makes the use of virtual images a tempting proposition; combine this with the fact that it is relatively easy to create an image or derivative of a base image and you start to see how this can lead to the creation of many images.

This type of proliferation, commonly referred to as virtual image sprawl, can quickly lead to rising management costs for software environments, so clearly, this is not a desirable outcome. In this sense, companies must keep a close eye on the number of overall virtual images in their inventory — easy to say but a good deal harder to do.

Many times image composers create virtual images so that users can simply deploy them and get to work. In order to do this, they usually build an image that statically captures the software components and a significant amount of configuration. While this may be the seemingly obvious approach, it can be a problem, especially when the configuration is specific to a narrow subset of use cases. When composers create images that contain components and a configuration specific to a particular use case, they decrease the likelihood that anyone can use the image for anything other than a very specific scenario.

This is the kind of approach that can quickly lead to image proliferation (virtual image sprawl). Figure 1 highlights this problem in the context of virtual images for middleware application environments.

Figure 1. Proliferation caused by increasing specificity
Proliferation caused by increasing specificity

Middleware application environments provide a good example to discuss virtual image challenges since they require various components and associated configurations. As shown in Figure 1, these environments include:

  • Operating systems
  • Middleware software
  • Applications
  • User-specific configuration

You can certainly decide to build a virtual image that includes everything up to the configuration that is specific to a user or use case; however, if that were the approach, you almost undoubtedly end up creating (and subsequently managing) many images.

On the other hand, you can build an image that contains only the operating system, but then you are delivering less value to your users since they need to install and configure quite a few other components.

The best answer here lies somewhere in the middle. In the case of building images for application middleware, the virtual image typically includes everything up through the middleware software components required by the application environment. By not including much user-specific content in the virtual image, you increase the likelihood that the image can support a number of scenarios which helps you keep your virtual image inventories at a reasonable level.

Although this approach helps you manage the number of virtual images while retaining a measure of value to the user, it does pose its own problem. By not directly building user configuration into the virtual image, you can decrease its consumability. Users have to activate the image and then perform a series of configuration steps before they can work with the environment. Depending on the complexity of the target environment, these steps may take a considerable amount of time and be hard to repeat from deployment to deployment. These problems frustrate virtual image users and dilute the value proposition promised by virtual images.

So you see, building useful, consumable virtual images while keeping inventory at reasonable levels is no easy task. Not easy, but not impossible.


A better way to build virtual images

Remember, the primary challenge in the use of virtual images is to keep the inventory at a manageable level while still providing considerable value to image users. To confront this challenge, you must take a measured approach to building your virtual images. To enable this type of approach, these are the fundamental questions:

  • What kind of content should you install directly into a virtual image?
  • How do you handle required content, such as user configuration, that you do not install directly into the virtual image?

Direct image installation content

The first step in building any virtual image is identifying the content that you want to burn into the image. This is not always an easy process and there is not always a right or wrong answer concerning whether or not you install a specific piece of content directly into the image.

In general, content that falls into the following categories is a good candidate for direct image installation:

Large components
You should install operating systems and other software of considerable size directly into your virtual image. If you require the installation of such components after activating the virtual image, you slow down the overall delivery of the environment, thus negating many of the benefits of a virtual image.
Common components
Identify the things that are common to most, if not all of your software environments, such as anti-virus software, monitoring agents, or office tools. By installing these components into the image, not only do you save time for your users, but you also decrease the chance that they end up with non-standard or non-compliant environments.
Time-intensive configuration
Configuration actions that take a considerable amount of time to perform are often good candidates to go directly into your images. This enables users to skip these actions and get to work quickly when using the image.

By burning content that fits into these three categories into your virtual images, you provide value by accelerating the delivery of an environment and removing configuration burden from the user.

Handling non-direct content: Not the ideal

As you can clearly see, these three categories represent only a subset of the overall needs of virtual image users. In particular, they do not address:

  • Users have to apply configurations specific to their usage scenarios.
  • Some configurations (IP addresses for example) change for every deployment.

You either should not or cannot burn this kind of configuration statically into the image, but that doesn't mean you should not attempt to account for it. One way to do this is to provide instructions or even scripts that guide users through the necessary post-activation configuration steps. This is not the ideal solution since it requires manual, post-deployment steps on the part of the image deployer.

Handling non-direct content: Creating dynamic virtual images

Instead of requiring users to perform post-deployment configuration steps, it is better to construct dynamic virtual images. The foundation of dynamic virtual images is the inclusion of a special set of activation scripts: These scripts perform dynamic configuration activities that need to occur for every deployment of the image. This can include updating the IP address, performing configuration actions on previously installed software, installing content that is small in size, or any number of other actions.

Your images need to surround these scripts with an activation framework. This activation framework:

  • Ensures the automated invocation of the scripts.
  • Provides a means to pass user input to the execution of the scripts.

Figure 2 illustrates the image-building approach just described.

Figure 2. An effective approach to image building
An effective approach to image building

As shown in Figure 2, start building images by installing your baseline software content into the virtual image. Next, provide dynamic configuration scripts surrounded by an activation framework, such as the one supporting the Open Virtualization Format.

The activation framework automates script invocation while defining a mechanism for user-supplied, deploy-time input. This careful approach balances static content versus dynamic configuration. In turn, this helps to keep virtual image inventory at manageable levels without sacrificing the ability to provide users with images that support their needs.


Before you implement this approach

It is one thing to outline a conceptual approach to building virtual images, but it is quite another to implement such an approach. You need tools that help you to do this and those tools need to address these questions at a minimum:

  • How can I make use of existing enterprise resources like servers and storage?
  • How can I handle the typical division of responsibilities within the organization when it comes to building software environments?
  • How can I effectively separate build time versus deploy time activation tasks for my images?

If you are looking for a tool to answer these questions, and more, look no further. The IBM Image Construction and Composition Tool is now part of IBM Workload Deployer 3.1, IBM SmartCloud Provisioning 1.2, and IBM SmartCloud Enterprise.


IBM Image Construction and Composition Tool

The IBM Image Construction and Composition Tool is a tool for building virtual images for deployment into cloud environments. Install the tool on a Linux system, and then connect it to one or more of your clouds (IBM Workload Deployer, SmartCloud Provisioning, or SmartCloud Enterprise) or directly to VMware ESX. The tool integrates with your existing cloud resources as the build environment for creating new images. If you prefer, you can also build images on SmartCloud Enterprise by deploying an instance of the IBM Image Construction and Composition Tool image provided on SmartCloud Enterprise.

Overview: Creating an image

To create an image using the tool, start with an operating system and customize it with software content, called bundles, and then generate a new image package.

Within the tool, multiple users can contribute content to use for image building. For example, an operating system expert can create an operating system definition based on your company's certified standard operating environment; a software specialist can define a bundle package that encapsulates installation and configuration actions for your required software content. This allows the person building an image to combine the necessary operating system and software components without needing in-depth knowledge of either the operating system or the software installation.

By making these tasks independent of each other — defining an operating system, creating a software bundle, and building an image — the tool makes it easy to map your organization's existing responsibilities and skills into the image building space:

  • Operating system specialists focus on the operating system layer.
  • Software specialists focus on adding components on top of the operating system.
  • Image builders focus on combining the operating system and software required to create new deployments in your enterprise cloud.

Overview: Managing installation and configuration

Within the IBM Image Construction and Composition Tool, you can manage installation and configuration tasks for your software.

When you create a base operating system image for use with the tool, it includes an activation framework that enables dynamic separation of build-time and deploy-time (configuration) actions.

When you define software bundles within the tool, you provide both installation tasks and configuration tasks independently of each other. The installation tasks run once, during the image build process, and the configuration tasks run for each deployment of the image. Separating these tasks gives you the ability to reduce the size of your image catalog and simplify image management.

In the next section, we'll explore the image-building process in more detail — creating a base operating system, defining software bundles, and generating image packages — and explain how the major workflows in the tool enable you to build consistent, customized virtual images that provide manageable separation of installation and user configuration tasks.


How to build your own images with ICCT

Figure 3 provides an overview of the interactions supported by the tool.

Figure 3. Image-building flows in the Image Construction and Composition Tool
Image-building flows in the Image Construction and Composition Tool

The tool is available with IBM Workload Deployer 3.1 or IBM SmartCloud Provisioning 1.2 (beta). Install it on a Linux set-up. You can also start an instance of the tool directly on IBM SmartCloud Enterprise. Image building relies on having access to a cloud environment (IBM Workload Deployer, IBM SmartCloud Provisioning, or SmartCloud Enterprise), or the tool also works directly with VMware ESX. These resources, or cloud providers, give the tool an environment in which to instantiate your operating system and install your custom bundle content.

Creating the base operating system

Image building always starts with the operating system. If you are connected to one of the supported clouds, you can start with an existing image available in the catalog, and import that operating system definition into the tool for further customization.

For IBM Workload Deployer VMware ESX images, you also have the option to build out your own custom operating system using your organization's operating system requirements. You can use the tool to take an existing virtual machine that already contains an installed operating system, and prepare it for use in the tool.

Steps 1 through 3 in Figure 3 illustrate this flow:

  1. ISO and Kickstart or operating system virtual machine.
  2. Create VM, install OS.
  3. Capture base OS image.

By consolidating the base operating system image creation in one area of the tool, you are able to take advantage of the existing skills within your organization to build an appropriate operating system layer based on your company’s standards, guaranteeing operating-system level consistency in your deployed environments.

In addition to defining the core operating system content and configuration, an operating system image must also contain an activation framework that enables customization of the operating system at deploy-time. This includes assigning IP addresses, configuring user accounts, setting up network interfaces, and more. The tool provides an activation framework that it includes in your virtual images.

Once you codify your standard operating system configuration as an image, you can use it as a base image and layer custom software bundles on top.

Defining software bundles

Within the tool, a bundle describes software that is available for use within an image. The bundle specification defines software pre-requisites and tasks for installing, configuring, and resetting the software.

Creating a bundle requires specialized knowledge of how the software works and is typically done by a software specialist, as shown in Step 4 in Figure 3 (4. Create bundle).

Inside of a bundle, the software specialist can define requirements, installation information, configuration tasks, and reset actions for the included software. It is a good practice to define any large or common components for your virtual images as software bundles. Middleware products — like IBM WebSphere® Application Server Community Edition or the IBM License Metric Tool Server or other common software packages, such as security compliance monitoring software or anti-virus agents — are all great candidates for bundles.

As you define a bundle in the tool, the user interface explicitly separates install and configuration parameters.

Install parameters
The install data for a bundle describes the tasks that should run when installing the bundle into a virtual image. Installation tasks only occur once, at image-building time, and the installed software becomes a permanent component of the image. In general, you should define any time-intensive or long-running tasks as installation tasks.

Some typical installation tasks include copying installation binaries to the image and running the installation programs for the software.

You can parameterize installation arguments in your bundle and expose those arguments as configurable input during the image building process. For example, if the installation location for a software package needs to be configurable at image build time, define an installation location argument and then use the value of that property in the bundle's installation scripts. By using parameterized installation arguments, you can use a single software bundle to build custom images with different installation requirements.

Configuration parameters
The configuration section of a bundle's definition provides deploy-time configuration options and activation scripts for the pre-installed software in the bundle. While the installation tasks run at image-build time, a software bundle's configuration tasks run during each image deployment.

If there are configuration parameters that image deployers need to customize in the deployed software — for example, administrative passwords or port definitions — you can mark those parameters as configurable inside the bundle definition.

During image deployment, the deployer can supply values for those configuration parameters and the bundle-supplied scripts within the activation engine can customize the software using the specified parameters. By using a combination of parameters and configuration scripts, you are able to define your software's instantiated profile at deploy-time rather than having to burn that custom configuration into the image.

This dynamic configuration capability significantly reduces the number of custom images that you need to create, drastically reducing the cost of managing your virtual image library.

Within the tool, bundles provide a flexible mechanism to capture and separate build-time actions and deploy-time tasks for your software. By encapsulating the software specialist's knowledge directly into the bundle, the image builder who uses the software bundle does not need to be an expert in installing and configuring the software. The image builder can simply choose the relevant operating system and software bundles and mix-and-match them to create customized virtual images.

Generating custom image packages

The IBM Image Construction and Composition Tool generates Open Virtual Format Archive (OVA) images for deployment using IBM Workload Deployer 3.1, IBM SmartCloud Provisioning 1.2 or VMware vSphere. The tool also creates new images on the IBM SmartCloud Enterprise public cloud.

The general steps that you, as the image builder, perform to create a new image package include:

  1. Extending a base operating system image
  2. Customizing the image with software bundles
  3. Synchronizing the image in the specified cloud provider
  4. Verifying the image configuration
  5. Capturing the image for use in future deployments

Steps 5 through 10 in Figure 3 illustrate this process. Because the operating system and software specialists have already captured their expertise within the tool, the image builder is able to quickly, easily combine operating system images and software bundles without needing in-depth knowledge of how to configure those components.

To build a new image, start by choosing a base operating system definition and then extending it to add software bundles (Step 5). Recall that software bundles can contain parameterized installation data. During the image extension stage, you can customize any configurable installation parameters for the bundles you are using in order to meet the requirements of the image you are building.

After saving the operating system and bundle configuration for the image, you need to synchronize your image (Step 6). During image synchronization, the tool copies the operating system image to the cloud provider and starts the operating system in a new virtual machine. Then the tool initiates the installation tasks for any software bundles you included. Remember, only the installation tasks run at this point. The tool builds the configuration tasks into the image at this point, but they do not run until image deployment time.

After the synchronization process is complete, it is a good practice to login to the virtual machine and verify the correct installation of the software you defined in the bundles.

Once synchronization is complete, capture your new image (Step 7):

  • If you are connected directly to IBM SmartCloud Provisioning, IBM Workload Deployer, or IBM SmartCloud Enterprise, the image is in the catalog ready to use for new deployments.
  • If you are using a local VMware ESX cloud provider, export your captured image from the tool as an OVA package, as shown in Figure 3, Step 8. This new OVA contains all of the data necessary to import your image into either IBM Workload Deployer or VMware vSphere so that you can deploy it to your private cloud.

When instantiating these images, regardless of the target cloud environment, you have the opportunity to customize any configuration data that you marked as configurable in the image's software bundles.


In conclusion

In this article, we discussed a better way to build virtual images. The focus is on keeping the image inventory at a manageable level while providing the most value to the most users (especially in terms of how widely used your image can be). The methodology we suggested is based on determining which content should be installed directly into the virtual image and which shouldn't; plus we offered ways to dynamically configure content installed in virtual images. In essence, we talked about how to build a custom cloud image that is highly reusable; one that provides the most amount of value with the least amount of work on the user's part.

And we offered the Image Construction and Composition Tool as a solution. The tool helps keep your virtual image library under control by providing a straightforward process for building useful, consumable virtual images. It lets you burn large, complex content, such as your certified operating system and other required content, directly into the images you build. The images built by the tool provide an activation engine for rich deployment-time customization. During activation, you can provide customization parameters for your environment based on the options defined in the image and its software bundles. This allows a single virtual image to take on many different forms during the activation process, thus allowing a single image to meet a wide variety of usage scenarios.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Cloud computing on developerWorks


  • Bluemix Developers Community

    Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

  • developerWorks Labs

    Experiment with new directions in software development.

  • DevOps Services

    Software development in the cloud. Register today to create a project.

  • Try SoftLayer Cloud

    Deploy public cloud instances in as few as 5 minutes. Try the SoftLayer public cloud instance for one month.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Cloud computing, WebSphere, Tivoli
ArticleID=642666
ArticleTitle=Establish a system to build custom virtual cloud images
publish-date=12072011