IBM SmartCloud Enterprise 2.0 introduces an image-importing feature that enables you to move your own Linux operating system image to the cloud, but the image-importing feature only supports the Linux operating system, not Windows® nor other UNIX operating systems. According to the Customizing Images and Software Bundles guide for SmartCloud Enterprise, the system only supports Red Hat or SuSE, but you can import other Linux operating system images as long as they meet conditions described in this article.

Most of the steps for the import process must be done using command lines and APIs. You need some knowledge of SmartCloud APIs, as well as minimal Linux OS installation and customization skills.

The Customizing Images and Software Bundles guide and REST API reference 2.0, available to SmartCloud users, provides you with good information, but I don't think they are sufficient to performing the task of importing a Linux OS effectively. This article provides you supplemental information.

Figure 1 provides an overview of image importation to SmartCloud Enterprise.

Let's categorize the important additional knowledge needed to perform this task into four general topic areas. Then we'll dive deeper into each one and I'll provide some tips for you to consider.

Four keys to importing a Linux OS

The four key steps of importing a Linux OS to SmartCloud Enterprise are:

1. Prepare your Linux OS image and metadata. You need to have a hypervisor to prepare your Linux OS image. Metadata has to be prepared manually, but you can reuse metadata of any public image available on SmartCloud as a template.
   
   
2. Store the image and metadata on your persistent storage. You can write your image and metadata by creating an instance that attaches a storage. You can upload your image and metadata over the network or you might want to create them on the instance.
   
   
3. Import your image from the storage as your private image. The command POST {baseURI}/offerings/image API is explained in order to support this task.
   
   
4. Create an instance from the image. The successfully imported image appears on your image list and you can create instances from it as you do with other private images.

The following references and components are recommended before you start:

• Customizing Images and Software Bundles guide and the reference guide REST API Reference 2.0 or the reference guide for the command-line tool you choose to use (the first two can be found on the Support tab documentation library of the SmartCloud portal).
• QEMU, the hypervisor and image manipulation tool.
• The ISO image of your target Linux OS.
• RealVNC VNC viewer.

Now let's take a closer look at each of the 4 steps. Tips are provided in each section.

Back to top

Prepare your Linux OS image and metadata

This section describes the following steps:

• Prepare the metadata template.
• Prepare the workbench instance.
• Install QEMU.
• Set up the VNC server.
• Prepare the OS image.
• Install the OS using QEMU.
• Minimum setup of the OS image.

Prepare a metadata template

The easiest way to prepare the set of metadata you need to import your image is to reuse that of a public image. If you are importing SuSE or its clone, reuse metadata from a public SuSE image. If your target is Red Hat or a clone of it such as CentOS, leverage the metadata from a public Red Hat image.

1. Since a public image is copy protected, you need to create an instance from the public image you want to reuse. Make sure you are creating a copper instance of the same processor architecture— for example, of a 32- or 64-bit image— as your target OS.
2. Once an instance of the public image is up, save your private image from the instance. Now you can set the CopyAllowed = Y flag on the private image from asset catalog screen of SmartCloud portal.
3. You have the link to the asset catalog on the header line of the Control panel tab of SmartCloud portal. Click the link and open the My Dashboard tab. You should see the list of your private image assets.
4. Click the pencil icon of the image you just saved, click the continue... link. A Copy Allowed flag appears at the bottom of the screen.
5. Change it to Y and click the Update button.
6. Create a persistent storage in ext3 format. 60GB is sufficient.
7. Copy the private image to the persistent storage using PUT {baseURI}/storage/{storage id} API.

Prepare the workbench instance

Figure 2 describes how to prepare the workbench instance.

After the copy is completed, create another instance attaching the storage. This is your workbench instance. Red Hat 6.0 64-bit is recommended, but you can use any Linux OS. Assume that /data is the mount point of the storage in the following description.

You will see a set of image and metadata files similar to this:

$ ls -l /data/image
total 3459192
-rw-rw-r--. 1 idcuser idcuser         451 Jan  3 03:14 BSS.zip
-rw-r--r--. 1 idcuser idcuser 64424509440 Jan  3 00:04 K7BqB_W2SOG3K@rUrln1EA.img
-rw-rw-r--. 1 idcuser idcuser         196 Jan  3 03:23 K7BqB_W2SOG3K@rUrln1EA.mf
-rwxr-xr-x. 1 idcuser idcuser        2013 Jan  3 00:04 K7BqB_W2SOG3K@rUrln1EA.ovf
-rwxr-xr-x. 1 idcuser idcuser       57830 Jan  3 00:04 RAM.zip
-rwxr-xr-x. 1 idcuser idcuser       20648 Jan  3 00:04 Terms.zip

The base name of .img, .mf, and .ovf files are different for each image. You can reuse RAM.zip, Terms.zip, and the .ovf file as they are. Other files can be deleted.

Install QEMU

Download the QEMU source tarball from the QEMU site and configure make and install:

wget http://wiki.qemu.org/download/qemu-1.0.tar.gz
tar -xzf qemu-1.0.tar.gz
cd qemu-1.0
./configure --target-list=x86_64-softmmu,i386-softmmu
make & install

You might want to add /usr/local/bin to your PATH environment variable because the previous instruction installs a set of QEMU tools on the directory.

Set up the VNC server

QEMU opens a virtual console on its own VNC server. You have to open a VNC connection to the XWindow of the instance using the VNC client on your PC and start another VNC client on the XWindow to connect to QEMU's VNC port.

Prepare the OS image

An OS image is a binary file which contains an HDD image of the installed Linux OS. It is also called a virtual disk since it can be driven by a hypervisor as an HDD. The easiest way to create an OS image is to install an OS on a virtual disk using a hypervisor and an ISO file of an OS install CD/DVD.

Install your OS using QEMU

You might want to do this step in a VNC session because you need to start the VNC client on XWindows to open the QEMU console as described earlier. You can use the RealVNC Viewer to connect to XWindows from your PC.

1. Create an empty image file in raw format:

   qemu-img create -f raw hdd.img 60G

   
   
2. Start the hypervisor with the image file and ISO file:

   qemu-system-i386 -m 1G -hda hdd.img -cdrom installer_image.iso -boot d

   
   
3. If you are installing a 64-bit OS, run qemu-system-x86_64 instead. To increase virtual memory, change the value after the -m flag.
4. You have to use a VNC client on the XWindow of the workbench instance to view the QEMU console. The TigarVNC viewer is available on RHEL6 in the submenu Applications > Internet for the purpose. Run the TigarVNC viewer and specify >localhost as the VNC server to connect to the QEMU console.
   
   Figure 3. Viewing a guest OS installation via VNC on a SmartCloud instance
   
   
   
5. It may be very slow to run the OS installer on QEMU because you don't have a KVM accelerator on the SmartCloud instance. You can do this step on your local PC, but you need to transfer a 10GB file over the Internet every time you make any change on the image until you finally complete the task.

After the installer has started, follow the installation instructions for your target OS, but make sure the following conditions are met:

• You can't use LVM; a recent Linux OS installer uses LVM by default. Select custom installation to avoid using LVM.
  
  
• Only /boot and / partition are allowed. Both of them must be in the primary partition and be in ext3 format. Again, you have to explicitly specify these parameters in the custom installation menu.
  
  
• No swap. You might want to increase virtual memory size when your installer hangs due to not having enough memory. You can use a text installer instead to decrease memory requirement of the installer, but some text installers do not provide a custom installation menu which is needed to support the first two conditions.

After successful installation of your target OS on the image file, reboot and login to check to see if the environment supports virtio as root user. The following two commands check if you have virtio kernel modules on /lib/modules and the initial RAM DISK image:

modprobe -l | grep virtio
gzip -dc /boot/initr*.img | cpio -t | grep virtio

If you miss virtio_pci or virtio_blk in the results of any of the previous code, you have to install the virtio kernel module and/or recreate the RAM DISK image. Otherwise your image can be imported but it will not boot.

The minimum setup of an OS image

The following is the minimum setup for an OS image before importing.

1. Create idcuser.

   groupadd -g 501 idcuser useradd -u 500 -g 501 idcuser passwd idcuser mkdir /home/idcuser/.ssh chown -R idcuser.idcuser /home/idcuser chmod 600 /home/idcuser/.ssh

   
   
2. Make idcuser sudo-enabled (/etc/sudoers).

   idcuser ALL=(ALL) NOPASSWD:ALL

   
   
3. Configure sshd (/etc/ssh/sshd_config).

I recommend you postpone this last step until you can get access to the instance as idcuser using an RSA key, like this:

PermitRootLogin no
PasswordAuthentication no
UsePAM no

Note: If you are importing a Linux OS other than a Red Hat or SuSE clone, something like Ubuntu, you need a way to convert network interface configuration. I recommend creating a reserved IP and setting up the IP address on the image for now. It enables you to connect to the instance anyway. You can customize your image after your image boots on SmartCloud.

Back to top

Store the image and metadata on your persistent storage

This section describes the following:

• Metadata preparation and structure.
• Disconnecting the storage.

Metadata preparation and structure

The following figure shows the metadata structure of an image on SmartCloud. Mandatory files are .ovf, .img, BSS.zip, RAM.zip, and .mf.

Reuse RAM.zip from a SmartCloud public image as described before, but you can look into the files and modify them if you are really sure about it.

You don't have to modify the .ovf file as long as you keep the names of these mandatory files. In other words, if you want to use your own name for your .img and .ovf file, you have to rewire the .ovf file.

1. Run the FileValidation.sh tool at the very end of the metadata preparation task to generate the .mf file. Since the .mf file contains a checksum of each file, you have to rerun the tool to regenerate the .mf file if you make any change after you create the .mf file.

   Here is an example of a BSS.xml in a BSS.zip. Consult the Customizing Images and Software Bundles guide for a detailed description of the file.

   <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <ns2:serviceoffering offeringtype="IMAGE" id="" xmlns:ns2="http://www.ibm.com/cloud/storage/xml"> <shortdesc>CentOS 6.1 32bit test</shortdesc> <longdesc>CentOS 6.1 32bit test</longdesc> <offeringfamilyid>1003</offeringfamilyid> <platformVersion>1.2</platformVersion> <attributes> <os>CentOS 6.1 32bit test/6.1</os> <allowedsize.32>COP32,BRZ32,SLV32,GLD32</allowedsize.32> </attributes> </ns2:serviceoffering>

   
   
2. The trick here is the <os> tag. You can specify the OS version number separated by "/" after the OS name.
3. Make sure the value of <offeringfamilyid> is 1003. Otherwise you are charged for your instance based on a different price table. The <allowedsize.32> tag's value is different when you are importing 64-bit OS image.
4. Create a BSS.zip file to include BSS.xml and store it on /data/image:

   zip /data/image/BSS.zip BSS.xml

   
   
5. Transfer your OS image to /data/image and rename it xxxx.img where xxxx is the same base name as that of xxxx.ovf.
6. Go to /data/image directory and run FileValidation.sh; select menu 2 to recreate the manifest file.

Disconnect the storage

You have to delete your workbench instance to disconnect the storage until a feature that lets you dynamically disconnect storage comes along. I strongly recommend that you take a private image of the instance before you delete it because you need to come back to this point to modify your image or metadata if your importing or provisioning fails. You can never be too optimistic: Your first try will likely fail for various reasons. Make sure that you can create an instance from the private image before you delete the workbench instance.

Back to top

Import your image from the storage as your private image

Import your image from the storage to a private image on SmartCloud using POST {baseURI}/offerings/image API. Take note of the image ID which is included in the response to your request. You need it when you check the status of your request or delete your image entry when it fails. Sometimes it is not on your private image list on the cloud portal, but it does exist and will consume your image count.

First attempt is likely to fail. Don't ask me why but don't worry. Take a deep breath, cross your fingers, and do it again before you give up.

Go back to the metadata preparation step upon failure; there is probably something wrong in your metadata.

Back to top

Create an instance from the image

This last section discusses provisioning the instance from your image and includes some Ubuntu support issues.

Provisioning your instance from your image

You can create your instance from your image from the portal. If you failed to create the instance, go back to image-creation step to confirm that all the conditions are met. Failure at this stage often comes from an incorrect image file format.

Try to get access as idcuser once your image comes up to be active. If your SSH client times out, your instance may not boot or may not configure the network interface. Go back to image-creation step and check the virtio configuration.

You might want to find out what's happened inside the instance. You can take your private image from the instance and copy it on storage, run the image with QEMU, and look into it.

If it booted, you should find some records on the syslog files.

If your SSH client receives authentication failure message, it might be an RSA key setup problem. You may be able to login as root. Check permission of files under /home/idcuser.

Once you're in

If you finally login as idcuser using the RSA key, keep the connection and configure the sshd_config file to not allow root access, password use, nor PAM use, and then restart sshd. Make sure you are still able to login as idcuser with the RSA key from another session.

If everything looks good, take your private image of the first generation instance and create a second generation instance from the image with a different IP address and RSA key.

Ubuntu support issues

The network configuration file of the Ubuntu OS is quite different from that of Red Hat. The provisioning engine of SmartCloud will set up a Red Hat-style config file which Ubuntu will not support. You need to implement following initd script for run level 3 through 5 to convert Red Hat-style config file into an Ubuntu-style one.

#!/usr/bin/perl

$INTERFACE="/etc/network/interfaces";
$NETCONFDIR="/etc/sysconfig/network-scripts/";

if ( (stat($INTERFACE))[9] > (stat($NETCONFDIR."ifcfg-eth0"))[9] ){
  exit 0;
}

system("/bin/sed -n s/HOSTNAME=//p /etc/sysconfig/network > /etc/hostname");
system("/bin/hostname `cat /etc/hostname`");

open(OUT,">$INTERFACE");
print OUT << "EOF";
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
EOF

open(TT,"ls ".$NETCONFDIR."ifcfg-eth?|");
while (<TT>){
  chop;
  $t=$n=$_;
  $n=~s/^.*\/ifcfg-//;
  open(IF,$t);
  while (<IF>){
    chop;
    if (/^IPADDR=(.*)$/){
      $addr=$1;
    } elsif (/^NETMASK=(.*)$/){
      $mask=$1;
    } elsif (/^GATEWAY=(.*)$/){
      $gw=$1;
    }
  }
  close(IF);

  @a=split(/\./,$addr);
  @m=split(/\./,$mask);

  for ($c=0; $c<4; $c++){
    $a=$a[$c]*1;
    $m=$m[$c]*1;
    $b[$c]=($a & $m) + (255 & ~ $m);
  }
  $bcast=sprintf("%d.%d.%d.%d",$b[0],$b[1],$b[2],$b[3]);

  print(OUT "iface $n inet static\n");
  print(OUT "    address $addr\n");
  print(OUT "    netmask $mask\n");
  print(OUT "    gateway $gw\n");
  print(OUT "    broadcast $bcast\n");
  print(OUT "\n");
}
close(TT);
close(OUT);

system("/etc/init.d/networking restart");
exit(0);

Back to top

In conclusion

The image-importing feature of IBM SmartCloud Enterprise 2.0 works. It supports Linux operating system images other than Red Hat or SuSE and you can even have a workbench instance on SmartCloud to create your image. But there are conditions and restrictions regarding the image format that make importing these OS images a bit more than entry-level-task difficult; the advice provided in this article should help you hurdle most of those obstacles.

Resources

Learn

• For more information on the technologies mentioned in this article: QEMU | RealVNC | Virtio.
  
  
• For more on how to perform tasks in the IBM Cloud, visit these resources:
  • Up and download files from a Windows instance.
  • Install IIS web server on Windows 2008 R2.
  • Create an IBM Cloud instance with the Linux command line.
  • Create an IBM Cloud instance with the Windows command line.
  • Extend your corporate network with the IBM Cloud.
  • High availability apps in the IBM Cloud.
  • Parameterize cloud images for custom instances on the fly.
  • Windows-targeted approaches to IBM Cloud provisioning.
  • Deploy products using rapid deployment service.
  • Integrate your authentication policy using a proxy.
  • Configure the Linux Logical Volume Manager.
  • Deploy a complex topology using a deployment utility tool.
  • Provision and configure an instance that spans a public and private VLAN.
  • Secure IBM Cloud access for Android devices.
  • Recover data in IBM SmartCloud Enterprise.
  • Secure virtual machine instances in the cloud.
  
  
• In the developerWorks cloud developer resources, discover and share knowledge and experience of application and services developers building their projects for cloud deployment.
  
  
• Find out how to access IBM SmartCloud Enterprise.
  
  

Get products and technologies

• See the product images available for IBM SmartCloud Enterprise.
  
  

Discuss

• Join a cloud computing group on developerWorks.
  
  
• Read all the great cloud blogs on developerWorks.
  
  
• Join the developerWorks community, a professional network and unified set of community tools for connecting, sharing, and collaborating.
  
  

About the author

Tomoyuki Niijima is an IT Architect who has been designing and implementing large-scale super-complex IT systems for the last 20 years. He co-authored three AIX Redbooks and published articles in several magazines, including SAP Professional Journal. He has also been active in the open source area. His focus in the last two years has been on cloud computing, especially on how complex components such as operating systems, middleware, applications, and operational tools can be deployed and managed rapidly, flexibly, and effectively.

Rate this article

Comments

Back to top

Table of contents

• Four keys to importing a Linux OS
• Prepare your Linux OS image and metadata
• Store the image and metadata on your persistent storage
• Import your image from the storage as your private image
• Create an instance from the image
• In conclusion
• Resources
• About the author
• Comments

Next steps from IBM

Learn more, join the cloud community

---

• Community: Cloud computing conversations led by IBM community members
• Buy: Try IBM software on IBM SmartCloud Enterprise

Dig deeper into Cloud computing on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers