Automate VM deployment

Save time and effort by launching batches of self-configuring VMs quickly

Sometimes you need to create an configure a bundle of virtual machines at the same time, but manually cloning and configuring the lot promises to be an unhappy task. In this article, see how to develop an automatic VM deployment solution so you can launch and activate batches of self-configuring VMs quickly. And as a bonus, you'll discover an approach that lets you run customized applications separately for each deployed virtual machine after system start.

Yong Kui Wang (wangyk@cn.ibm.com), Software Engineer, IBM

Yong Kui Wang is a software engineer at IBM China Systems and Technology Laboratories and is currently focusing on system management software development. He is also interested and experienced in Linux technologies, vitalization, and Web 2.0 technologies.



Jie Li (lijiesh@cn.ibm.com), Software Engineer, IBM

Jie Li works as a software engineer at IBM China Systems and Technology Laboratories. He is experienced in J2EE development and currently focuses on system management software development. In his spare time, his interests include JavaScript engine and off-line Web application development (just like Gears).



04 March 2009

Also available in Russian Japanese

In software development and testing, you often need to create a bunch of virtual machines at the same time. You have to find and copy a template virtual image, modify its display name and hardware settings, and solve the network-interface hardware-address conflict. Add the configuring of network settings, hostname, and domain settings—and it quickly becomes very labor intensive.

In this article, see how to develop an automatic virtual machine deployment solution that will help you deploy and activate batches of self-configuring virtual machines quickly and automatically. This approach will also let you run customized applications separately for each deployed virtual machine after system start.

Architecture and workflow of automatic VM deployment

The automatic virtual machine deployment solution we're offering mainly consists of two parts - the Virtual Machine Deployment Manager (VDM) and Virtual Machine Configuration Manager (VCM).

The Virtual Machine Deployment Manager is responsible for handling user's VM deployment requests such as cloning virtual images, configuring VM hardware settings, and registering the VM to the VM hypervisor

The Virtual Machine Configuration Manager is installed in the virtual image used as a VM template; it will configure the VM automatically after system start. Figure 1 shows the architecture and workflow of our automatic VM deployment solution.

Figure 1. Architecture of the automatic VM deployment framework
Architecture of the automatic VM deployment framework

As you can see, the VDM consists of three parts:

  • The VDM Engine, the core of the VDM.
  • VM deployment tasks are a description file of a certain VM deployment task, sort of a configuration file for users.
  • VM Template Definitions (VTD) files define the VM templates information such as location and description of the VM template. A VM template is a virtual machine that will be used as source VM for cloning.

For the most part, the VM deployment process consists following steps:

  1. VDM reads in the VM deployment task.
  2. VDM finds the corresponding VM template by going though the directory containing VTD files and cloning the VM template to the destination specified in the deployment task file.
  3. After cloning the source VM, VDM collects all configuration data, archives it into an ISO file, and then copies it to the target location of the cloned VM.
  4. VDM modifies the hardware settings of the cloned VM based on the configurations defined in VM deployment task file; it will create a CD-ROM device to mount the ISO file created in step 3.
  5. The last deployment step is calling the VM hypervisor utilities to register the VM to VM hypervisor server and then to power it on.

Step 5 is optional; the user can choose not to do so by configuring the VM deployment task file. The VM hypervisor can be VMware server, Xen, etc. The sample implementation of this article uses a VMware server as the VM hypervisor server.

The VCM is installed in the VM template. After a system starts, it will start automatically and search the CD containing the configuration data, then launch the configuration applications on it to get all predefined work done.


Create self-configuring template VMs

It should be obvious that to make a VM self-configuring after system boot, the source VM used as template must be self-configuring. To create a self-configuring VM template, the source VM should be installed with VCM after initial creation as shown in Figure 2.

Figure 2. Creating a self-configuring VM template by installing VCM
Creating a self-configuring VM template by installing VCM

The main function of the VCM is to find the VM configuration CD labeled VMCONIFG and run the configuration applications on it after system start. Because the source VM may be Linux®, Windows®, or other operating systems, the VCM of different operating systems are different. In the sample code for this article, the VCMs for Linux and Windows are provided.

VM Configuration Manager for Linux

The Linux VM Configuration Manager will be registered as a Linux service after install. It will run during the first system boot. When starting, it will perform the configuration and then unregister and remove itself after the configuration is done.

To install VCM for Linux, copy the VCM package into the source VM, decompress it to some directory, and run VCM installer install.sh. Listing 1 shows you how install.sh works.

Listing 1. Sample code segment of VCM installer install.sh
...
VCM_HOME=$(cd $(dirname $0);pwd)
start=`getStartNum`
stop=`getStopNum`
sed -e "s!^VCM_HOME=.*!VCM_HOME=$VCM_HOME!;\
  s!^# chkconfig: 35!# chkconfig: 35 $start $stop!"\
  "$VCM_HOME/vmconfigmgr" > "/etc/init.d/vmconfigmgr"
chmod +x "/etc/init.d/vmconfigmgr"
echo "Register service vmconfigmgr"
chkconfig --add vmconfigmgr
if [ $? -eq 0  ];then
    echo "Install Complete Successfully!"
else
    echo "Install Failed."
Fi

Because all configuration applications and data are saved on the VM configuration CD, the VCM will first find the CD and mount it to /media/VMCONFIG before it can do any configuration. Listing 2 shows how the Linux VCM finds the VM configuration CD after system start.

Listing 2. Sample code of finding and mounting VM configuration CD by VCM
LABEL="VMCONFIG"
VM_CFG_DIR=/media/$LABEL
for device in `dmesg | grep "^.*:.*CD-ROM" | awk -F':' '{print $1}'`
do
    volumeName=`volname "/dev/$device" | awk '{print $1}'`
    PrintString "CD-ROM Drive: $device | Label: $volumeName"	
    if [ "$volumeName" == "$LABEL" ];then
        PrintString "VM Configuration CD-ROM is: /dev/$device"
        # Mount the CD-ROM 
        mkdir -p "$VM_CFG_DIR"        
        mount -t iso9660 -o ro,nosuid,nodev,utf8,uid=0 "/dev/$device" "$VM_CFG_DIR" 
        break
    fi
done

VM Configuration Manager for Windows

After install, the Windows VM Configuration Manager will create a scheduled task that will run once after system start. Like VCM for Linux, Windows VCM will also remove the scheduled task after the first run.

To install Windows VCM, copy the Windows VCM package into the Windows VM template, unzip it to some directory, and run install.bat. Listing 3 shows you how install.bat works.

Listing 3. Sample code of installing Windows VCM
@echo off

set VCM_HOME=%~dp0
REM Create scheduled task to run VM Configuration Manager when system boots
schtasks /create /tn "VMCONFIG"  /tr %VCM_HOME%vmconfigmgr.bat /sc onstart /ru "System"

@echo on

Similar to the Linux VCM, the Windows VCM also needs to know the drive letter of the VM configuration CD. The Windows VCM uses a WMI script to do this. Listing 4 shows the VB script.

Listing 4. Sample code of VB script used by Windows VCM to get the drive letter of VM configuration CD
Function getDriveLetter(label)
	Const wbemFlagReturnImmediately = &h10
	Const wbemFlagForwardOnly = &h20
	strComputer="."
	driveLetter=""
	Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\CIMV2")
	Set colItems = objWMIService.ExecQuery("SELECT * FROM Win32_CDROMDrive", "WQL", _
                                          wbemFlagReturnImmediately + 
                                           wbemFlagForwardOnly)
	For Each objItem In colItems
		WScript.Echo "Drive: " & objItem.Drive & "  VolumeName: " 
                      & objItem.VolumeName  & "  Status: " & objItem.Status
		If StrComp(objItem.VolumeName, label) = 0 Then
			driveLetter=objItem.Drive
			WScript.Echo "driveLetter: " & driveLetter
			Exit For 
		End If  
	Next
	getDriveLetter=driveLetter
End Function

drive=getDriveLetter("VMCONFIG")

After installing the VCM into the source VM, the source VM becomes a self-configuring VM template. Shut it down and save it at a certain location.


Create VM template definition files

The VM templates can be stored at different location, so the VM Deployment Manager needs a mechanism to know where a certain VM template is. The VM Template Definition file is designed for this; it describes the properties of a VM template.

A single VTD file points to a single VM template. All VTD files are stored in the vtds folder of the VDM. The file name of a VDT file looks like OSName.vtd (in which OSName is used as a key to distinguish one VM template from another). For example, you can use SLES10SP2.vtd as the file name of a VTD pointing to a SLES 10 SP2 VM template. Listing 5 shows a sample VTD of a SLES10 SP2 VM template.

Listing 5. Sample VTD of SLES 10 SP2 VM template
# VM Template Definition File
# OS Type of the template VM, possible values are Windows, Linux.
OSType=Linux
# Description of the VM template.
Description=SuSE Linux Enterprise Server 10 SP2
# Directory where the VM template locates in.
SRCVMDIR=/local/vmware/SuSE Linux Enterprise Server 10 SP2

Create VM deployment tasks

Now that all the preparation work has been done, you can start to create a VM deployment task. A VM deployment task is defined by a VM deployment task file. One VM deployment task defines the all the configuration command of how to deploy a certain VM. All these configuration commands of a VM deployment task can fall into different groups:

  • a general VM configuration,
  • network configurations,
  • customized user configurations, and
  • a VM register/start configuration.

Listings 6, 7, 8, and 9 show the configuration settings separately of deployment task sles10sp2vm.task.

Listing 6. General VM configuration settings in VM Deployment Task File
# Task description
TaskDescription=SLES 10 VM used for BVT
# OS name of the VM to be created, a file with the name OSName.vtd should has been 
# defined in vtds directory.
OSName=SLES10SP2
# Size of the memory in MB that should be assigned to the VM
MemorySize=1300
# Disk size in GB of the first hard disk, if the number specified is not specified or 
# is smaller than the disk size of template VM, it will has same disk size as 
# template VM.
DiskSize=10
# Display Name of the VM.
DisplayName=SLES10SP2-143-50
# The destination directory of the VM to be created to.
Destination=/vmware/SLES10SP2-143-50
# Overwrite specifies if overwrite the existing VM.
OverwriteOnExist=Yes

These general settings define the OS type, memory size, disk size, and target location of the VM to be created. It also allows users to choose if to overwrite when there is already a virtual machine at the destination location.

Listing 7. Network configuration settings in VM Deployment Task File
# Specify if to configure the network during the first boot of the VM. Yes to configure,
# or No for not configure. 
# If No is specified, all properties start with ConfigNet will be ignored.
ConfigNet=yes
# Network connection type of Virtual Ethernet card eth0, possible values are Bridged,
# NAT and HostOnly. Default value is Bridged if not specified.
ConfigNet.ConnectType=NAT
# Mac address of the ethernet card, possible values are:
#    1) auto                 the mac will be generated automatically.
#    2) xx:xx:xx:xx:xx:xx        user defined mac address. (For VMware, the valid range
#       is 00:50:56:00:00:00 to 00:50:56:3F:FF:FF)
ConfigNet.MacAddress=00:50:56:3A:01:02
# Network configuration mode, dhcp or static.
ConfigNet.Mode=static
# IP address to be used for the VM.
ConfigNet.IPAddress=192.168.143.50
ConfigNet.Netmask=255.255.255.0
ConfigNet.Gateway=192.168.143.2
ConfigNet.Hostname=sles10vm-143-50
ConfigNet.Domain=ibm.com
ConfigNet.PrimaryDNS=9.181.32.72
ConfigNet.SecondDNS=9.181.2.101

When deploying, the network configuration information will be collected by VDM and will be saved to the VM configuration ISO file so the information can be accessed when the VM starts.

Listing 8. Customized user configuration settings in VM Deployment Task File
# Folder holding user's configuration data. The content of this folder will be archived
# into an ISO file, with label VMCONFIG and be mounted in the VM CD-ROM.
UserConfigDataDir=/root/mydata/
# The application users want to run after system boot and network configuration.
# It will also be archive to the VMCONFIG ISO file.
UserConfigApp=/root/myapp.sh

All user configuration data will also be collected and saved to VM configuration ISO file.

Listing 9. VM register/start configuration settings in VM Deployment Task File
# Register VM to local vmware server or not. Yes for register, false for not register.
# Default value is No.
RegisterVM=Yes
# Parameters used to register VM. Not needed by VMware Server 1.x.
RegisterVM.HostURL=https://localhost:8333
RegisterVM.Username=root
RegisterVM.Password=password
# Power on VM after deployment or not. Yes for power on, No for not power on.
# Default value is false.
PowerOn=Yes

These parameters are used to register or power on a VM after deployment. For VMware Server 2.0, you must provide the server URL, username, and password.


Deploy VMs

Since both VM templates and the VM deployment task file are ready, it's time to deploy. The VM deployment process includes

  • parsing the settings of the VM deployment task,
  • finding the right VM template and cloning it to the target folder,
  • generating an ISO file containing the configuration data,
  • modifying the hardware configurations of the new VM and mounting the ISO file as a CD, and at last,
  • registering and starting the VM if RegisterVM and PowerOn properties are set to Yes in the VM deployment task file.

In the sample code for this article, the VM Deployment Manager is designed as untar-and-run so you don't need additional steps to install it, just untar it to your favorite place (for example, /opt/vmdeploymgr). However, to register and start the new VM, the server installed VDM should have some VM hypervisor utilities installed. The VM hypervisor utilities should be able to be called by the VDM to register and power on a VM. In the sample code, the VDM is installed on the same server as the VMware server.

VM deployment options

The VDM provides three deployment options:

  1. Deploying a single VM.
  2. Deploying several VM deployment tasks using a task list.
  3. Deploying all VM deployment tasks stored in a folder.

To support this, three options of command vmdeploymgr.sh are provided which is the main application of the VDM.

  1. Deploy a single VM deployment task.

The VDM allows you to point to a specified VM deployment task file by using the -f option as shown: localhost:/opt/vmdeploymgr/bin # ./vmdeploymgr.sh -f ../tasks/sles10sp2.task.

  1. Deploy several VM deployment tasks using a task list.

You can also list all the VM deployment tasks in the task list file and ask the VDM to process all the tasks in that list sequentially. The task list file is a plain text file. Listing 10 shows a sample task list file containing three VM deployment tasks, the last task (win2003vm2.task) will not be processed by the VDM because of having been commented out.

Listing 10. Sample VM Deployment Task List File
/opt/vmdeploymgr/tasks/sles10sp2vm.task
/opt/vmdeploymgr/tasks/rhel5u2vm.task
/opt/vmdeploymgr/tasks/win2003vm.task
#/opt/vmdeploymgr/tasks/win2003vm2.task

Option -t is used for deploying a VM deployment task list. The following listing shows the sample command:

localhost:/opt/vmdeploymgr/bin # ./vmdeploymgr.sh -t ../tasks/vmtasklist
  1. Deploy all VM deployment tasks stored in a folder.

The third option of VM deployment is to deploy all the task files in specified folder. Command option -d is used for this; if -d is given, the VDM will search all files with extension .task in a given folder then process these tasks. The following listing shows the sample command:

localhost:/opt/vmdeploymgr/bin # ./vmdeploymgr.sh -d ../tasks

The VM configuration CD

The VM configuration CD is a very important component of the automatic VM deployment solution because it contains both configuration data from the VDM and users. It makes it possible to create virtual machines of specific, different operating systems with different configurations using the same VM template. In another words, you only need one VM template for each operating system -- this saves time and disk space.

The CD is simulated by an ISO file generated by the VDM according to the VM deployment task and will be mounted to a virtual CD-ROM so it can hold far more data than a floppy disk. The VM configuration CD gives you the flexibility of running whatever application you want after the VM starts. The VM configuration ISO file is named as VMCONFIG.iso and the label of it is VMCONFIG. Figure 3 shows the file structure of the VM configuration CD for Linux. The file structure of that for Windows is similar.

Figure 3. File structures of the VM configuration CD for Linux VM
File structures of the VM configuration CD for Linux VM

As shown in Figure 3, folder vdm mainly contains predefined configuration tasks defined by the VDM (for example, network configuration) while folder usr contains user customized configuration data defined in the VM deployment task file. File run.conf is used to tell the VCM what application needs to be run on the CD. Listing 11 shows its sample content.

Listing 11. Sample content of run.conf
# Configuration application to perform predefined VM Deployment Manager configuration,
# for example, network configuration.
VDM_CFG_APP=vdm/run.sh
# User defined configuration application that will run after system boot.
USR_CFG_APP=usr/myapp.sh

When the deployed VM is powered on, the VCM will start and try to locate the VM configuration CD. Once it finds the CD, it will read the content of run.conf and then execute the applications defined by VDM_CFG_APP and USR_CFG_APP sequentially. If user does not specify any customized configuration in the VM deployment task file, the property USR_CFG_APP will not be set in run.conf.

A new CD-ROM will be created for the cloned VM to mount the VM configuration CD and the VDM will search the IDE buses (first, then SCSI buses) of the VM to find an idle bus for the CD-ROM. Once it finds one, it will modify the VM configuration file, create a new CD-ROM, and then mount the CD to it.

Register and power on the VM

To register a VM to VM hypervisor server and start it, the VDM needs the support of some VM hypervisor utilities. For VMware Server 2.0, the main tool is vmrun; for VMware Server 1.0, it's vmware-cmd. These tools will be installed by default when you install a VMware server.

Listing 12 shows the sample code demonstrating how to register and start a VM using vmware-cmd; Listing 13 shows the sample code demonstrating how to register and start a VM using vmrun:

Listing 12. Sample code of registering and starting VM using vmware-cmd
vmware-cmd  -s register "$vmxfile"
vmware-cmd  "$vmxfile" start hard
Listing 13. Sample code of registering and starting VM using vmrun
vmrun -T server -h "${hostURL}/sdk" -u "$username" -p "$password"\ 
      register "[$datastoreName] $vmxfile"
vmrun -T server -h "${hostURL}/sdk" -u "$username" -p "$password"\
      start "[$datastoreName] $vmxfile"

Summary

You can see with the automatic virtual machine deployment solution we've described in this article, the deployment and initial configuration of virtual machines become very easy. All you need to do is to write down the configurations in the VM deployment task files and the VM Deployment Manager will then have all the virtual machines deployed and configured automatically for you.

This solution is also very flexible too -- you can run your own applications after the deployed VM starts. For example, you can ask the VM to run an application to download and install the new software build automatically after system start by setting USR_CFG_APP in the VM deployment task.

All in all, this solution should help you save a lots of time and work more efficiently when deploying and managing a stable of virtual machines.


Downloads

DescriptionNameSize
VM Configuration Manager, Linux1vmconfigmgr-linux.zip5KB
VM Configuration Manager, Windows2vmconfigmgr-windows.zip3KB
VM Deployment Manager3vmdeploymgr.zip14KB

Notes

  1. The VM Configuration Manager, Linux configuration manager package was tested on SLES 10 SP1, SP2, and Red Hat Enterprise Linux Server (RHEL) 5.2.
  2. The VM Configuration Manager, Windows configuration manager package was tested on Windows Server 2003 Enterprise Edition and Window Server 2008 Enterprise Edition.
  3. The VM Deployment Manager is the deployment manager package; this development platform is SUSE Linux Enterprise Server (SLES) SP1, and it was tested with VMware Server 2.0 (installed on SLES10 SP1) and VMware Server 1.0.4 (installed on SLES10).

Resources

Learn

Get products and technologies

  • With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Linux on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Linux
ArticleID=374187
ArticleTitle=Automate VM deployment
publish-date=03042009