In software development and testing, you often need to create a bunch of virtual machines at the same time. You have to find and copy a template virtual image, modify its display name and hardware settings, and solve the network-interface hardware-address conflict. Add the configuring of network settings, hostname, and domain settings—and it quickly becomes very labor intensive.

In this article, see how to develop an automatic virtual machine deployment solution that will help you deploy and activate batches of self-configuring virtual machines quickly and automatically. This approach will also let you run customized applications separately for each deployed virtual machine after system start.

Architecture and workflow of automatic VM deployment

The automatic virtual machine deployment solution we're offering mainly consists of two parts - the Virtual Machine Deployment Manager (VDM) and Virtual Machine Configuration Manager (VCM).

The Virtual Machine Deployment Manager is responsible for handling user's VM deployment requests such as cloning virtual images, configuring VM hardware settings, and registering the VM to the VM hypervisor

The Virtual Machine Configuration Manager is installed in the virtual image used as a VM template; it will configure the VM automatically after system start. Figure 1 shows the architecture and workflow of our automatic VM deployment solution.

As you can see, the VDM consists of three parts:

• The VDM Engine, the core of the VDM.
• VM deployment tasks are a description file of a certain VM deployment task, sort of a configuration file for users.
• VM Template Definitions (VTD) files define the VM templates information such as location and description of the VM template. A VM template is a virtual machine that will be used as source VM for cloning.

For the most part, the VM deployment process consists following steps:

1. VDM reads in the VM deployment task.
2. VDM finds the corresponding VM template by going though the directory containing VTD files and cloning the VM template to the destination specified in the deployment task file.
3. After cloning the source VM, VDM collects all configuration data, archives it into an ISO file, and then copies it to the target location of the cloned VM.
4. VDM modifies the hardware settings of the cloned VM based on the configurations defined in VM deployment task file; it will create a CD-ROM device to mount the ISO file created in step 3.
5. The last deployment step is calling the VM hypervisor utilities to register the VM to VM hypervisor server and then to power it on.

Step 5 is optional; the user can choose not to do so by configuring the VM deployment task file. The VM hypervisor can be VMware server, Xen, etc. The sample implementation of this article uses a VMware server as the VM hypervisor server.

The VCM is installed in the VM template. After a system starts, it will start automatically and search the CD containing the configuration data, then launch the configuration applications on it to get all predefined work done.

Back to top

Create self-configuring template VMs

It should be obvious that to make a VM self-configuring after system boot, the source VM used as template must be self-configuring. To create a self-configuring VM template, the source VM should be installed with VCM after initial creation as shown in Figure 2.

The main function of the VCM is to find the VM configuration CD labeled VMCONIFG and run the configuration applications on it after system start. Because the source VM may be Linux®, Windows®, or other operating systems, the VCM of different operating systems are different. In the sample code for this article, the VCMs for Linux and Windows are provided.

VM Configuration Manager for Linux

The Linux VM Configuration Manager will be registered as a Linux service after install. It will run during the first system boot. When starting, it will perform the configuration and then unregister and remove itself after the configuration is done.

To install VCM for Linux, copy the VCM package into the source VM, decompress it to some directory, and run VCM installer install.sh. Listing 1 shows you how install.sh works.

...
VCM_HOME=$(cd $(dirname $0);pwd)
start=`getStartNum`
stop=`getStopNum`
sed -e "s!^VCM_HOME=.*!VCM_HOME=$VCM_HOME!;\
  s!^# chkconfig: 35!# chkconfig: 35 $start $stop!"\
  "$VCM_HOME/vmconfigmgr" > "/etc/init.d/vmconfigmgr"
chmod +x "/etc/init.d/vmconfigmgr"
echo "Register service vmconfigmgr"
chkconfig --add vmconfigmgr
if [ $? -eq 0  ];then
    echo "Install Complete Successfully!"
else
    echo "Install Failed."
Fi

Because all configuration applications and data are saved on the VM configuration CD, the VCM will first find the CD and mount it to /media/VMCONFIG before it can do any configuration. Listing 2 shows how the Linux VCM finds the VM configuration CD after system start.

LABEL="VMCONFIG"
VM_CFG_DIR=/media/$LABEL
for device in `dmesg | grep "^.*:.*CD-ROM" | awk -F':' '{print $1}'`
do
    volumeName=`volname "/dev/$device" | awk '{print $1}'`
    PrintString "CD-ROM Drive: $device | Label: $volumeName"	
    if [ "$volumeName" == "$LABEL" ];then
        PrintString "VM Configuration CD-ROM is: /dev/$device"
        # Mount the CD-ROM 
        mkdir -p "$VM_CFG_DIR"        
        mount -t iso9660 -o ro,nosuid,nodev,utf8,uid=0 "/dev/$device" "$VM_CFG_DIR" 
        break
    fi
done

VM Configuration Manager for Windows

After install, the Windows VM Configuration Manager will create a scheduled task that will run once after system start. Like VCM for Linux, Windows VCM will also remove the scheduled task after the first run.

To install Windows VCM, copy the Windows VCM package into the Windows VM template, unzip it to some directory, and run install.bat. Listing 3 shows you how install.bat works.

@echo off

set VCM_HOME=%~dp0
REM Create scheduled task to run VM Configuration Manager when system boots
schtasks /create /tn "VMCONFIG"  /tr %VCM_HOME%vmconfigmgr.bat /sc onstart /ru "System"

@echo on

Similar to the Linux VCM, the Windows VCM also needs to know the drive letter of the VM configuration CD. The Windows VCM uses a WMI script to do this. Listing 4 shows the VB script.

Function getDriveLetter(label)
	Const wbemFlagReturnImmediately = &h10
	Const wbemFlagForwardOnly = &h20
	strComputer="."
	driveLetter=""
	Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\CIMV2")
	Set colItems = objWMIService.ExecQuery("SELECT * FROM Win32_CDROMDrive", "WQL", _
                                          wbemFlagReturnImmediately + 
                                           wbemFlagForwardOnly)
	For Each objItem In colItems
		WScript.Echo "Drive: " & objItem.Drive & "  VolumeName: " 
                      & objItem.VolumeName  & "  Status: " & objItem.Status
		If StrComp(objItem.VolumeName, label) = 0 Then
			driveLetter=objItem.Drive
			WScript.Echo "driveLetter: " & driveLetter
			Exit For 
		End If  
	Next
	getDriveLetter=driveLetter
End Function

drive=getDriveLetter("VMCONFIG")

After installing the VCM into the source VM, the source VM becomes a self-configuring VM template. Shut it down and save it at a certain location.

Back to top

Create VM template definition files

The VM templates can be stored at different location, so the VM Deployment Manager needs a mechanism to know where a certain VM template is. The VM Template Definition file is designed for this; it describes the properties of a VM template.

A single VTD file points to a single VM template. All VTD files are stored in the vtds folder of the VDM. The file name of a VDT file looks like OSName.vtd (in which OSName is used as a key to distinguish one VM template from another). For example, you can use SLES10SP2.vtd as the file name of a VTD pointing to a SLES 10 SP2 VM template. Listing 5 shows a sample VTD of a SLES10 SP2 VM template.

# VM Template Definition File
# OS Type of the template VM, possible values are Windows, Linux.
OSType=Linux
# Description of the VM template.
Description=SuSE Linux Enterprise Server 10 SP2
# Directory where the VM template locates in.
SRCVMDIR=/local/vmware/SuSE Linux Enterprise Server 10 SP2

Back to top

Create VM deployment tasks

Now that all the preparation work has been done, you can start to create a VM deployment task. A VM deployment task is defined by a VM deployment task file. One VM deployment task defines the all the configuration command of how to deploy a certain VM. All these configuration commands of a VM deployment task can fall into different groups:

• a general VM configuration,
• network configurations,
• customized user configurations, and
• a VM register/start configuration.

Listings 6, 7, 8, and 9 show the configuration settings separately of deployment task sles10sp2vm.task.

# Task description
TaskDescription=SLES 10 VM used for BVT
# OS name of the VM to be created, a file with the name OSName.vtd should has been 
# defined in vtds directory.
OSName=SLES10SP2
# Size of the memory in MB that should be assigned to the VM
MemorySize=1300
# Disk size in GB of the first hard disk, if the number specified is not specified or 
# is smaller than the disk size of template VM, it will has same disk size as 
# template VM.
DiskSize=10
# Display Name of the VM.
DisplayName=SLES10SP2-143-50
# The destination directory of the VM to be created to.
Destination=/vmware/SLES10SP2-143-50
# Overwrite specifies if overwrite the existing VM.
OverwriteOnExist=Yes

These general settings define the OS type, memory size, disk size, and target location of the VM to be created. It also allows users to choose if to overwrite when there is already a virtual machine at the destination location.

# Specify if to configure the network during the first boot of the VM. Yes to configure,
# or No for not configure. 
# If No is specified, all properties start with ConfigNet will be ignored.
ConfigNet=yes
# Network connection type of Virtual Ethernet card eth0, possible values are Bridged,
# NAT and HostOnly. Default value is Bridged if not specified.
ConfigNet.ConnectType=NAT
# Mac address of the ethernet card, possible values are:
#    1) auto                 the mac will be generated automatically.
#    2) xx:xx:xx:xx:xx:xx        user defined mac address. (For VMware, the valid range
#       is 00:50:56:00:00:00 to 00:50:56:3F:FF:FF)
ConfigNet.MacAddress=00:50:56:3A:01:02
# Network configuration mode, dhcp or static.
ConfigNet.Mode=static
# IP address to be used for the VM.
ConfigNet.IPAddress=192.168.143.50
ConfigNet.Netmask=255.255.255.0
ConfigNet.Gateway=192.168.143.2
ConfigNet.Hostname=sles10vm-143-50
ConfigNet.Domain=ibm.com
ConfigNet.PrimaryDNS=9.181.32.72
ConfigNet.SecondDNS=9.181.2.101

When deploying, the network configuration information will be collected by VDM and will be saved to the VM configuration ISO file so the information can be accessed when the VM starts.

# Folder holding user's configuration data. The content of this folder will be archived
# into an ISO file, with label VMCONFIG and be mounted in the VM CD-ROM.
UserConfigDataDir=/root/mydata/
# The application users want to run after system boot and network configuration.
# It will also be archive to the VMCONFIG ISO file.
UserConfigApp=/root/myapp.sh

All user configuration data will also be collected and saved to VM configuration ISO file.

# Register VM to local vmware server or not. Yes for register, false for not register.
# Default value is No.
RegisterVM=Yes
# Parameters used to register VM. Not needed by VMware Server 1.x.
RegisterVM.HostURL=https://localhost:8333
RegisterVM.Username=root
RegisterVM.Password=password
# Power on VM after deployment or not. Yes for power on, No for not power on.
# Default value is false.
PowerOn=Yes

These parameters are used to register or power on a VM after deployment. For VMware Server 2.0, you must provide the server URL, username, and password.

Back to top

Deploy VMs

Since both VM templates and the VM deployment task file are ready, it's time to deploy. The VM deployment process includes

• parsing the settings of the VM deployment task,
• finding the right VM template and cloning it to the target folder,
• generating an ISO file containing the configuration data,
• modifying the hardware configurations of the new VM and mounting the ISO file as a CD, and at last,
• registering and starting the VM if RegisterVM and PowerOn properties are set to Yes in the VM deployment task file.

In the sample code for this article, the VM Deployment Manager is designed as untar-and-run so you don't need additional steps to install it, just untar it to your favorite place (for example, /opt/vmdeploymgr). However, to register and start the new VM, the server installed VDM should have some VM hypervisor utilities installed. The VM hypervisor utilities should be able to be called by the VDM to register and power on a VM. In the sample code, the VDM is installed on the same server as the VMware server.

VM deployment options

The VDM provides three deployment options:

1. Deploying a single VM.
2. Deploying several VM deployment tasks using a task list.
3. Deploying all VM deployment tasks stored in a folder.

To support this, three options of command vmdeploymgr.sh are provided which is the main application of the VDM.

1. Deploy a single VM deployment task.

The VDM allows you to point to a specified VM deployment task file by using the -f option as shown: localhost:/opt/vmdeploymgr/bin # ./vmdeploymgr.sh -f ../tasks/sles10sp2.task.

2. Deploy several VM deployment tasks using a task list.

You can also list all the VM deployment tasks in the task list file and ask the VDM to process all the tasks in that list sequentially. The task list file is a plain text file. Listing 10 shows a sample task list file containing three VM deployment tasks, the last task (win2003vm2.task) will not be processed by the VDM because of having been commented out.

/opt/vmdeploymgr/tasks/sles10sp2vm.task
/opt/vmdeploymgr/tasks/rhel5u2vm.task
/opt/vmdeploymgr/tasks/win2003vm.task
#/opt/vmdeploymgr/tasks/win2003vm2.task

Option -t is used for deploying a VM deployment task list. The following listing shows the sample command:

localhost:/opt/vmdeploymgr/bin # ./vmdeploymgr.sh -t ../tasks/vmtasklist

3. Deploy all VM deployment tasks stored in a folder.

The third option of VM deployment is to deploy all the task files in specified folder. Command option -d is used for this; if -d is given, the VDM will search all files with extension .task in a given folder then process these tasks. The following listing shows the sample command:

localhost:/opt/vmdeploymgr/bin # ./vmdeploymgr.sh -d ../tasks 

The VM configuration CD

The VM configuration CD is a very important component of the automatic VM deployment solution because it contains both configuration data from the VDM and users. It makes it possible to create virtual machines of specific, different operating systems with different configurations using the same VM template. In another words, you only need one VM template for each operating system -- this saves time and disk space.

The CD is simulated by an ISO file generated by the VDM according to the VM deployment task and will be mounted to a virtual CD-ROM so it can hold far more data than a floppy disk. The VM configuration CD gives you the flexibility of running whatever application you want after the VM starts. The VM configuration ISO file is named as VMCONFIG.iso and the label of it is VMCONFIG. Figure 3 shows the file structure of the VM configuration CD for Linux. The file structure of that for Windows is similar.

As shown in Figure 3, folder vdm mainly contains predefined configuration tasks defined by the VDM (for example, network configuration) while folder usr contains user customized configuration data defined in the VM deployment task file. File run.conf is used to tell the VCM what application needs to be run on the CD. Listing 11 shows its sample content.

# Configuration application to perform predefined VM Deployment Manager configuration,
# for example, network configuration.
VDM_CFG_APP=vdm/run.sh
# User defined configuration application that will run after system boot.
USR_CFG_APP=usr/myapp.sh

When the deployed VM is powered on, the VCM will start and try to locate the VM configuration CD. Once it finds the CD, it will read the content of run.conf and then execute the applications defined by VDM_CFG_APP and USR_CFG_APP sequentially. If user does not specify any customized configuration in the VM deployment task file, the property USR_CFG_APP will not be set in run.conf.

A new CD-ROM will be created for the cloned VM to mount the VM configuration CD and the VDM will search the IDE buses (first, then SCSI buses) of the VM to find an idle bus for the CD-ROM. Once it finds one, it will modify the VM configuration file, create a new CD-ROM, and then mount the CD to it.

Register and power on the VM

To register a VM to VM hypervisor server and start it, the VDM needs the support of some VM hypervisor utilities. For VMware Server 2.0, the main tool is vmrun; for VMware Server 1.0, it's vmware-cmd. These tools will be installed by default when you install a VMware server.

Listing 12 shows the sample code demonstrating how to register and start a VM using vmware-cmd; Listing 13 shows the sample code demonstrating how to register and start a VM using vmrun:

vmware-cmd  -s register "$vmxfile"
vmware-cmd  "$vmxfile" start hard

vmrun -T server -h "${hostURL}/sdk" -u "$username" -p "$password"\ 
      register "[$datastoreName] $vmxfile"
vmrun -T server -h "${hostURL}/sdk" -u "$username" -p "$password"\
      start "[$datastoreName] $vmxfile"

Back to top

Summary

You can see with the automatic virtual machine deployment solution we've described in this article, the deployment and initial configuration of virtual machines become very easy. All you need to do is to write down the configurations in the VM deployment task files and the VM Deployment Manager will then have all the virtual machines deployed and configured automatically for you.

This solution is also very flexible too -- you can run your own applications after the deployed VM starts. For example, you can ask the VM to run an application to download and install the new software build automatically after system start by setting USR_CFG_APP in the VM deployment task.

All in all, this solution should help you save a lots of time and work more efficiently when deploying and managing a stable of virtual machines.

Back to top

Downloads

Information about download methods

Notes

1. The VM Configuration Manager, Linux configuration manager package was tested on SLES 10 SP1, SP2, and Red Hat Enterprise Linux Server (RHEL) 5.2.
2. The VM Configuration Manager, Windows configuration manager package was tested on Windows Server 2003 Enterprise Edition and Window Server 2008 Enterprise Edition.
3. The VM Deployment Manager is the deployment manager package; this development platform is SUSE Linux Enterprise Server (SLES) SP1, and it was tested with VMware Server 2.0 (installed on SLES10 SP1) and VMware Server 1.0.4 (installed on SLES10).

Resources

Learn

• "The Linux operating system as a managed object" (developerWorks, July 2008) takes a look at two major challenges -- deployment and management in an increasingly virtualized world -- by looking at the Linux operating system as a virtual, managed object.
  
  
• See all Linux tips on developerWorks.
  
  
• In the developerWorks Linux zone, find more resources for Linux developers, including Linux tutorials, as well as our readers' most recent favorite Linux articles and tutorials (For instance, for this article, search developerWorks for "systems administration" within "Linux.")
  
  
• Stay current with developerWorks technical events and Webcasts.
  
  
• Browse the technology bookstore for books on these and other technical topics.
  
  

Get products and technologies

• Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
  
  
• With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.
  
  

Discuss

• Get involved in the developerWorks community through our developer blogs, forums, podcasts, and community topics in our new developerWorks spaces.
  
  

About the authors

Comments (Undergoing maintenance)

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Architecture and workflow of automatic VM deployment
• Create self-configuring template VMs
• Create VM template definition files
• Create VM deployment tasks
• Deploy VMs
• Summary
• Downloads
• Resources
• About the authors
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

My developerWorks community

Dig deeper into Linux on developerWorks

Rate a product. Write a review.

Write a review for Rational products today!

Special offers