IBM SmartCloud™ init (an IBM SmartCloud enablement bundle) is a first boot script inspired by the open source cloud-init package. It was developed in the context of the integration between the Image Construction and Composition Tools (ICCT) and IBM SmartCloud Provisioning 1.2, as part of the ICCT enablement bundle and was later extended to become a key component in IBM SmartCloud Provisioning 2.1. All of the intermediate and advanced images in SmartCloud Provisioning 2.1 include cloud-init.

IBM SmartCloud init is a simple, stand-alone component that is executed at virtual machine (VM) boot time to consume configuration data formatted as multipart Multipurpose Internet Mail Extensions (MIME). IBM SmartCloud init takes this input and looks at the content type associated with each part. If the content type is known, IBM SmartCloud init invokes the related consumer.

Possible use cases leveraging the IBM SmartCloud init script

The first use case and associated MIME type was to support Open Virtualization Format (OVF) images that ICON produced. The current implementation handles just one content type (application/ovf-param) and one consumer to handle OVF metadata, saving the related part as OVF-env.xml for the Activation Engine (AE).

You can use IBM SmartCloud init to bootstrap a cloud instance that performs a list of actions such as:

• Provide the transport layer for the OVF environment file
• Install a list of Red Hat Package Managers (RPMs) at boot time
• Execute commands at boot time
• Download and execute scripts at boot time (useful for virtual-to-virtual conversion)
• Process IWD add-ons and script packages

Back to top

IBM SmartCloud init deliverables

scp-cloud-init is a simple component available for Linux and Windows. The files are part of SmartCloud Provisioning 2.1 and can be retrieved directly from the server: http://<SmartCloudProvisioning server>/downloads/cloud-init/.

scp-cloud-init in Linux contains the file scp-cloud-init. The easiest way to install this file in an image using run level 3 and 5 is to run the following command:

cp -f scp-cloud-init.sh /etc/init.d/scp-cloud-init 2>/dev/null
chkconfig scp-cloud-init on

If you're mounting the disk and can't use chkconfig, you can manually generate the rc#.d links with the commands shown in Listing 1.

	
if [ -d /etc/rc3.d ]; then
   ln -fs ../init.d/scp-cloud-init /etc/rc3.d/S11scp-cloud-init
   chmod 755 /etc/rc3.d/S11scp-cloud-init 2>/dev/null
fi
if [ -d /etc/init.d/rc3.d ]; then
   ln -fs ../scp-cloud-init /etc/init.d/rc3.d/S11scp-cloud-init
   chmod 755 /etc/init.d/rc3.d/S11scp-cloud-init 2>/dev/null
fi
if [ -d /etc/rc5.d ]; then
   ln -fs ../init.d/scp-cloud-init /etc/rc5.d/S11scp-cloud-init
   chmod 755 /etc/rc5.d/S11scp-cloud-init 2>/dev/null
fi
if [ -d /etc/init.d/rc5.d ]; then
   ln -fs ../scp-cloud-init /etc/init.d/rc5.d/S11scp-cloud-init
   chmod 755 /etc/init.d/rc5.d/S11scp-cloud-init 2>/dev/null
fi 
# The S## above should be after the "network" but before any activation.* (AE) scripts.

scp-cloud-init in Windows contains the files scp-cloud-init.cmd and scp-cloud-init.vbs. To install them in an image to run at startup:

1. Copy the scp-cloud-init.* files to a directory such as C:\windows\setup\ibm.
2. Add the script to run at startup using the Group Policy Editor by running gpedit.msc.
3. Click Computer Configuration > Windows Settings > Scripts, and then double-click Startup.
4. Click Add, and then type c:\Windows\Setup\IBM\scp-cloud-init.cmd as the program (no parameters).
5. Click OK, then OK again to exit the startup-script window, and then quit the gpedit application.

The previous method for adding a registry key to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run (shown below) only seems to execute the script when the administrator logs in and doesn't do it automatically when the system starts.

"reg add HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run 
   /f /v a-cloud-init 
   /d c:\windows\setup\ibm\scp-cloud-init.cmd"

scp-cloud-init is available as an RPM package in Linux and as an .msi file in Windows.

Back to top

User data input format

You configure IBM SmartCloud init's behavior through user data, which users can provide at instance launch. Do that using the --user-data or --user-data-file argument to iaas-run-instances or by passing it in the IBM SmartCloud Provisioning web console in the Launch Instance panel. User data can be retrieved in either of two ways: by invoking the URL http://169.254.169.254/latest/user-data or by retrieving it directly from disks (Windows) or CD-ROM (Linux). User data that IBM SmartCloud init will process must be in one of the following types:

• MIME multipart archive. This list of rules is applied to each part of this multipart file. Using a multipart MIME file, users can specify more than one type of data. For example, they could specify both a user-data script and a cloud-config type.
• User-data script. This script begins with: #! or Content-Type: text/x-shellscript and will be executed at the rc.local-like level during first boot. (rc.local-like means "very late in the boot sequence.")
• OVF parameters. This type begins with #ovf-parameters or Content-Type: text/ovf-parameters. This content is the ovf-env.xml file, which can be used to pass the ovf-env.xml file to the Virtual Solutions Activation Engine (VSAE, under /opt/ibm/ae/AP) for processing the activation scripts.
• Cloud config data. This type begins with #cloud-config or Content-Type: text/cloud-config. This content is "cloud config" data and can be used to install RPMs or invoke commands or scripts.
• IWD script packages. Type type begins with #cloud-iwd or Content-Type: text/cloud-iwd. Use this content to download and install IWD script packages.
• Init volumes. This type begins with #init-volumes or Content-Type: text/init-volumes. Use this content to tell how to mount attached volumes and optionally format them first.

OVF parameter syntax

The OVF parameter syntax, implemented in IBM SmartCloud Provisioning 1.2 for Linux and in IBM SmartCloud Provisioning 2.1 for Windows, contains the ovf-env.xml file that is passed to the VSAE under /opt/ibm/ae/AP for Linux and C:\windows\setup\ibm\AP on Windows. Listing 2 shows the OVF environment file.

	
<?xml version="1.0" encoding="utf-8" ?>
   <ovfenv:Environment xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1"
      ovfenv:id="LINUX">
   <ovfenv:PropertySection>
   <ovfenv:Property ovfenv:key="ConfigTEMAgent.TEMServerHostname" 
      ovfenv:value="tem- server1.romleba.it.ibm.com"/>
   <ovfenv:Property ovfenv:key="ConfigTEMAgent.TEMServerPort" 
      ovfenv:value="9090"/>
   </ovfenv:PropertySection></ovfenv:Environment>

With Linux, IBM SmartCloud also supports the presence of an optional /opt/ibm/scp/default-ovf-env.xml file. This file can contain default settings that are overridden on an AE-class (script) basis by the user data-provided ovf-env.xml file. If no OVF parameter part is included in the user data, this default file is used as the ovf-env.xml file for the AE. The default-ovf-env.xml file allows cloud image providers to use the AE scripts, such as one to re-install an IBM Tivoli Monitoring (ITM) agent, during the first boot of a VM without requiring the right parameters to be passed down from the higher OVF management layers during every deployment, for example, if the /opt/ibm/scp/default-ovf-env.xml file contained the following code:

<?xml version="1.0" encoding="ASCII"?>
<ovfenv:Environment xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" 
   ovfenv:id="RHEL">
<ovfenv:PropertySection>
<ovfenv:Property ovfenv:key="ConfigPWD_ROOT.password" 
   ovfenv:value="passw0rd"/>
<ovfenv:Property 
   ovfenv:key="ITMInstall.nfspath" 
   ovfenv:value="idpx1mgr.cloud9.ibm.com:/software/monitoring/ITM"/>
<ovfenv:Property ovfenv:key="ITMInstall.scriptname" 
   ovfenv:value="scripts/V3_install.guest.lz.sh"/>
</ovfenv:PropertySection>
</ovfenv:Environment>

If the user data's ovf-env.xml file specified any references of ovfenv:key="ConfigPWD_ROOT.anything", then that line in the default-ovf-env.xml file is not merged into the user data's ovf-env.xml file. However, if the user data does not reference ITMInstall, then those two lines are merged into the user data's ovf-env.xml file. It is intended to support this default-ovf-env.xml functionality in Windows, when possible.

User-data script

A user-data script is a simple way to add script that is executed at the rc.local-like level during first boot. Currently, for Linux scp-cloud-init, these commands are run at the end of the /etc/rc.local script, and then removed from that script afterwards.

Cloud config syntax

Cloud config is a simple way to perform certain actions through user data. Using cloud-config syntax, users can specify certain things in a human-friendly format. For example, with cloud-config syntax, users can run a few commands on first boot. The output of these commands appear in console output:

runcmd:
- [ wget, "http://slashdot.org", -O, /tmp/index.html ]
- [ sh, -xc, "echo $(date) ': hello world!'" ]

Similarly, users can install packages at boot time as follows:

packages:
- pkg1
- pkg2

IWD script packages

The IWD script provides a way to download IWD add-ons and script packages located on an HTTP server.

In IWD 3.1.0.x, these add-ons and script packages are located in the directory /drouter/ramdisk2/mnt/raid-volume/raid0/rainmaker-scripts; but for scp-cloud-init, they have to be available on an HTTP server.

The add-ons and script packages are provided in a single list and organized based on the order defined in the IWD user interface (UI). To install add-ons and script packages, use the code in Listing 3.

	
Content-type: text/cloud-iwd
Cookie:	SimpleToken=mytoken
IPs: 9.123.141.38,10.10.0.11,172.16.0.11,9.168.105.232
scripts:
- https://SmartCloudProvisioning/resources/addons/5/defaultadduserwin.zip 
   [USERNAME="pino",PASSWORD="mypass"]
- https://9.168.58.195/resources/addons/5/defaultadduserwin.zip
   [USERNAME="pino",PASSWORD="mypass"]
- https://9.152.137.41/resources/scripts/7/scriptArchive.zip 
   [HelloWorld1="hello-peter"]

The package is a .zip file that contains a cbscript.json file and the script to be invoked. Listing 4 provides an example of cbscript.json.

	
[
 {
   "description": "RM05015",
   "command": "sh /tmp/defaultadduser/adduser",
   "log": "/tmp/defaultadduser",
   "location": "/tmp/defaultadduser",
   "timeout": 120000,
   "type": "ADDON_USER",
   "keys":
     [
       {
         "scriptkey": "USERNAME",
         "scriptvalue": "",
         "scriptdefaultvalue": ""
       },
       {
         "scriptkey": "PASSWORD",
         "scriptvalue": "",
         "scriptdefaultvalue": "",
         "type": "password"
       }
     ]
 }
]

Listing 5 shows an example of the script.

	
 #!/bin/sh
 # Default add-on script to create a user.  This script must be run as root!
 ...
 ...
 # User add-on scripts may define parameters for any required information.
 # Parameter values are passed as environment variables when the script
 # is run.  This default user add-on script accepts the following parameters: 
 #
 # USERNAME - User name of the user to be added. 
 #
 # PASSWORD - Password to be set for the new user.  If not supplied, no
 #   password is set for the user.
 echo "*** creating user ${USERNAME}"
 useradd -m ${USERNAME}
 if [ -n "$PASSWORD" ]
 then
   echo "*** setting password for user ${USERNAME}"
   case $(uname) in
       AIX)
           echo "${USERNAME}:${PASSWORD}" | /usr/bin/chpasswd -c
           ;;
       *)
           echo ${PASSWORD} | /usr/bin/passwd --stdin ${USERNAME}
           ;;
   esac
 fi

Part handler syntax (not implemented)

The part-handler is written to a file in /var/lib/cloud/data based on its file name. The part-handler must be Python code that contains a list_types method and a handle_type method. Once the section is read, the list_types method is called. This method must return a list of MIME types that this part-handler handles. IBM SmartCloud Provisioning Cloud-init will then call the handle_type method once at the beginning, once per part received, and once at the end. The begin and end calls are to allow the part handler to perform initialization or tear-down.

Listing 6 shows an example part-handler.

	
#part-handler
def list_types():
   # return a list of mime-types that are handled by this module
   return(["text/plain", "text/go-cubs-go"])
def handle_part(data,ctype,filename,payload,frequency):
   # data: the cloudinit object
   # ctype: '__begin__', '__end__', or the specific mime-type of the part
   # filename: the filename for the part, or dynamically generated part if
   #           no filename is given attribute is present
   # payload: the content of the part (empty for begin or end)
   # frequency: the frequency that this cloud-init run is running for
   #            this is either 'per-instance' or 'always'.  'per-instance'
   #            will be invoked only on the first boot.  'always' will
   #            will be called on subsequent boots.
   if ctype == "__begin__":
      print "my handler is beginning, frequency=%s" % frequency
      return
   if ctype == "__end__":
      print "my handler is ending, frequency=%s" % frequency
      return
   print "==== received ctype=%s filename=%s ====" % (ctype,filename)
   print payload
   print "==== end ctype=%s filename=%s" % (ctype, filename)

Init volumes syntax (implemented in Linux)

This syntax tells how scp-cloud-init should mount the attached volumes, specified by their label field in the IBM SmartCloud Provisioning iaas-describe-volumes application programming interface (API). That label value is a random 4-byte value written into the Master Boot Record's Disk Signature field when the volumes are created, which allows this script to differentiate between multiple attached volumes. Here are two example mount commands:

#init-volumes
label=5000ff02 format=ext3 mount_dir=/test_output format_label=test-output
label=2303ff04 mount_dir=/test_input

Each volume is a new line, with each parameter a <parameter-name>=<value> pair separated by one or more spaces. The two required parameters are label and mount_dir. Optional parameters are part_num for the partition number inside the volume disk (the default is no partition), format to specify the file system type with which to format this volume before mounting it, and format_label to specify the file system label to be applied while formatting.

Example user data

It is important to recognize that the #-prefixed headers imply that all of the user data contains only a single section type, as there is no end-boundary marker with it. Listing 7 provides an example of a single-section OVF parameter.

	
#ovf-parameters
<?xml version="1.0" encoding="ASCII"?>
<ovfenv:Environment xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" 
   ovfenv:id="RHEL">
  <ovfenv:PropertySection>
    <ovfenv:Property ovfenv:key="ConfigPWD_ROOT.password" ovfenv:value="PASSWORD"/>
  </ovfenv:PropertySection>
</ovfenv:Environment>

Similarly, a single user-data script looks like this:

#!/bin/bash
echo "I am a user-data script that runs at the end of /etc/rc.local"

To use multiple parts, use the Content-Type: headers and the appropriate end boundary. Although the default boundary is --::::::::::IBM-Tivoli-SmartCloud-Provisioning-Boundary::::::::::, you can also redefine that boundary in the header of the first part with boundary="<boundary-goes-here>". Note that a -- prefix is appended, per the MIME Request for Comments (RFC), to the boundary specified. Also note that although the MIME RFC allows it, this script will not support changing the boundary beyond the header of the first part.

Listing 8 provides examples of multipart user data.

	
Content-type: multipart/mixed; 
   boundary="::::::::::IBM-Tivoli-SmartCloud-Provisioning-Boundary::::::::::"
--::::::::::IBM-Tivoli-SmartCloud-Provisioning-Boundary::::::::::
Content-type: text/plain
--::::::::::IBM-Tivoli-SmartCloud-Provisioning-Boundary::::::::::
Content-type: application/ovf_parameters
<?xml version="1.0" encoding="utf-8" ?>
<ovfenv:Environment xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1"  
   ovfenv:id="LINUX">
<ovfenv:PropertySection>
<ovfenv:Property ovfenv:key="ConfigITMAgent.server" 
   ovfenv:value="srv1.romelab.it.ibm.com"/>
<ovfenv:Property ovfenv:key="ConfigITMAgent.port" ovfenv:value="11111"/>
<ovfenv:Property ovfenv:key="ConfigDB2.db2admin_password" 
   ovfenv:value="password"/>
<ovfenv:Property ovfenv:key="ConfigDB2.port" ovfenv:value="22222"/>
<ovfenv:Property ovfenv:key="ITMConfig.server name" 
   ovfenv:value="srv2.romelab.it.ibm.com"/>
<ovfenv:Property ovfenv:key="ITMConfig.port" ovfenv:value="333333"/>
</ovfenv:PropertySection></ovfenv:Environment>
--::::::::::IBM-Tivoli-SmartCloud-Provisioning-Boundary::::::::::
Content-type: application/user_data
my user data ....
--::::::::::IBM-Tivoli-SmartCloud-Provisioning-Boundary::::::::::
Content-type: text/x-shellscript
dir c:\temp
copy c:\temp\web_console.log c:\temp\web_console.bak 
echo pino > c:\temp\test
--::::::::::IBM-Tivoli-SmartCloud-Provisioning-Boundary::::::::::
Content-type: text/cloud-config

--::::::::::IBM-Tivoli-SmartCloud-Provisioning-Boundary::::::::::
Content-type: text/cloud-iwd
Cookie:	SimpleToken=mytoken
IPs: 9.123.141.38,10.10.0.11,172.16.0.11,9.168.105.232
scripts:
- https://SmartCloudProvisioning/resources/addons/5/defaultadduserwin.zip 
   [USERNAME="pino",PASSWORD="mypass"]
- https://9.168.58.195/resources/addons/5/defaultadduserwin.zip 
   [USERNAME="pino",PASSWORD="mypass"]
- https://9.152.137.41/resources/scripts/7/scriptArchive.zip [HelloWorld1="hello-peter"]
--::::::::::IBM-Tivoli-SmartCloud-Provisioning-Boundary::::::::::

MIME-Version: 1.0
Content-type: multipart/mixed; boundary="XXXXboundary text"
echo "This is a multipart message in MIME format"
--XXXXboundary text
Content-type: text/cloud-config;
runcmd:
- [ echo, "I am in cloud-config" ]
- [ curl, -o, /tmp/index.html, "http://slashdot.org" ]
- [ echo, "I am a following print command" ]
--XXXXboundary text
Content-type: wtf/fake;
Verifying handling of unrecognized MIME part
--XXXXboundary text
Content-type: application/ovf_parameters;
<?xml version="1.0" encoding="ASCII"?>
<ovfenv:Environment xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" 
   ovfenv:id="RHEL">
<ovfenv:PropertySection>
<ovfenv:Property ovfenv:key="ConfigPWD_ROOT.password" 
   ovfenv:value="PASSWORD"/>
</ovfenv:PropertySection>
</ovfenv:Environment>
--XXXXboundary text--

Back to top

Serviceability

IBM SmartCloud init creates a log file in the following directory:

• Windows: %TEMP%/scp-cloud-init-<date><time>.log

  Note that when this script is executed using Group Policy at start-up, %TEMP% resolves to C:\Windows\Temp. When executed by the Administrator account, it goes to C:\Users\Administrator\Local Settings\Temp. The messages from this script are also logged in the Windows event logger and can be viewed from the Event Viewer Control Panel item (the source is "SCP cloud init").

• Linux: /opt/ibm/scp/scp-cloud-init.log

These log files are purged when the script detects that it is running in a new VM so as not to keep the potentially sensitive data from the parent VM the VM image came from.

Back to top

Design details

The main design point for the IBM SmartCloud init extension is to make it pluggable by providing specific consumers of specific content types, one for each item listed in this article. One idea is to deploy the consumers in a specific folder and let IBM SmartCloud init scan that folder and look at the available consumers to understand which consumers can manage which content types. In this way, you can have code on the guest operating system, which downloads and installs things (just like BigFix does today with the Fixlets approach).

Back to top

Conclusion

Several open questions remain. First, should IBM make IBM SmartCloud init an alternative to the AE. Is there anything the AE does and that IBM SmartCloud init cannot do? IBM SmartCloud init does not replace OVF and VSAE but rather provides a flexible, extensible, and dynamic mechanism that you can exploit as you choose. Consider that IBM SmartCloud Provisioning 1.2 created the first management discipline to support OVF and hooked into the UI, the image metadata, and of course the guest via an OVF part handler. You can add TEM, ITM, and many other Tivoli disciplines as well as things more typical from the open source cloud-init.

Similarly, should users leverage this tool just for the initial image creation, at deployment time, or both? The IBM SmartCloud init component can drive the installation of each bundle (or Fixlet, chef recipe, and so on) before invoking the capture image API. There is no need for it to do VSAE cleanup for capture, because the scp-cloud-init script detects when this is a new instance (by detecting a media access control address change) and drives cleanup automatically before forwarding the ovf-env.xml file into it.

Resources

Learn

• In the developerWorks cloud developer resources, discover and share knowledge and experience of application and services developers building their projects for cloud deployment.
  
  
• Follow developerWorks on Twitter.
  
  
• Watch developerWorks demos ranging from product installation and setup demos for beginners to advanced functionality for experienced developers.
  
  

Get products and technologies

• Learn more about the Ubuntu cloud-init package.
  
  
• Evaluate IBM products in the way that suits you best: Download a product trial, try a product online, use a product in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.
  
  

Discuss

• Get involved in the developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.
  
  

About the authors

Rate this article

Comments

Back to top

Table of contents

• Possible use cases leveraging the IBM SmartCloud init script
• IBM SmartCloud init deliverables
• User data input format
• Serviceability
• Design details
• Conclusion
• Resources
• About the authors
• Comments

Next steps from IBM

Learn more, join the cloud community

---

• Community: Cloud computing conversations led by IBM community members
• Buy: Try IBM software on IBM SmartCloud Enterprise

Dig deeper into Cloud computing on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers