Use Apache Deltacloud to administer multiple instances with a single API

Discover how to "abstract" cloud providers to administer VMs at various providers

Become familiar with the core concepts of Apache Deltacloud, a web-based framework that lets you use a single web application or API to administer instances on many different cloud providers. Learn how to install and set up Deltacloud; discover its API basic features that let you script provisioning for virtual machines on a host of different cloud providers. Explore examples of using Deltacloud to start instances on Amazon EC2 and get information about instances on IBM® SmartCloud™.

Nathan A. Good, Senior Consultant and Freelance Developer, Freelance Developer

Nathan GoodNathan A. Good lives in the Twin Cities area of Minnesota. Professionally, he does software development, software architecture, and systems administration. When he's not writing software, he enjoys building PCs and servers, reading about and working with new technologies, and trying to get his friends to make the move to open source software. He's written and co-written many books and articles, including Professional Red Hat Enterprise Linux 3, Regular Expression Recipes: A Problem-Solution Approach, and Foundations of PEAR: Rapid PHP Development.



17 December 2012

Also available in Chinese Japanese

Apache Deltacloud, which graduated from the Apache Software Foundation Incubator in October 2011, is a web-based framework that allows you to use a single web application or application programming interface (API) to administer instances in many different cloud providers, such as Amazon Elastic Compute Cloud (Amazon EC2), Windows Azure™, OpenStack, and IBM SmartCloud. This article shows you how to install and set up Deltacloud on your computer. See examples of using Deltacloud to start instances on Amazon EC2 and get information about instances on IBM SmartCloud.

Prerequisites

To get the most out of this article, you should sign up for accounts on these services so that you can create and administer machine instances. See Resources to learn more about creating accounts.

Meet, install, and run Apache Deltacloud

Deltacloud is a web application implemented in the Ruby programming language that you can install and execute locally in your environment. Deltacloud uses the notion of a driver, which is an adapter that connects to different cloud vendors. Specify which driver you want Deltacloud to use, which then allows Deltacloud to send commands to different cloud vendors on your behalf.

Instead of interacting directly with the different cloud vendors, you can interact with Deltacloud. This gives you the ability to create portable scripts for performing operations such as creating machine instances. You can write scripts that call the Representational State Transfer (REST)-ful APIs that Deltacloud exposes. In other words, you can use Deltacloud like a facade for any scripts you want to write to provision machines in the cloud.

High-level architecture

Install Deltacloud on a server accessible to your various desktops. From the desktops, you can browse to the web application, or you can execute scripts that call the RESTful API. Deltacloud uses the driver to transform your request to a request that the cloud vendor to which you're attached understands. Figure 1 provides a high-level diagram showing this interaction.

Figure 1. Deltacloud high-level architecture
Image showing Deltacloud's high-level architecture

Next, learn how to install Deltacloud on a system and use it to work with virtual machine (VM) instances hosted by Amazon or IBM.

Installing Deltacloud

Installing the Deltacloud web application is relatively simple once you have all of the dependencies resolved, which are fairly significant. To install Deltacloud, you need the following tools (see Resources for links):

  • Ruby 1.8.7+
  • RubyGems 1.3.7+
  • G++
  • libxml2 and libxml2-dev
  • libxslt and libxslt-dev
  • Several Ruby gems, including thin, sinatra, rack-accept, rest-client, sinatra-content-for, and nokogiri

Installing all of these dependencies is much easier on a distribution of Linux® that handles your dependencies for you, because any one of these dependencies can have other dependencies. On a standard system with little installed, you can spend most of the time getting the system prepared for the next step.

Install Deltacloud on Linux

This article demonstrates how to install the prerequisites using Ubuntu Linux and the Windows® operating system. For information on installing Deltacloud on other platforms, see Resources.

To install the prerequisites on Ubuntu Linux, type the following commands:

$ sudo apt-get update
$ sudo apt-get install ruby1.9.1-full
$ sudo apt-get install g++ libxml libxml2-dev libxslt libxslt-dev
$ gem install thin sinatra rack-accept rest-client sinatra-content-for nokogiri

The Advanced Package Manager (APT) commands (apt-get) take care of handling any other dependencies that are required, which makes it a fairly easy way to get everything up and running.

Install Deltacloud on Windows

To install the prerequisites on Windows, you first need to install Ruby. See Resources for links to download a Ruby installer for Windows. The Ruby Installer installs itself in a directory such as C:\Ruby192, where 192 is the version of Ruby you're installing.

Then, install the DevKit package (see Resources). After you have installed DevKit and the Ruby Installer, you can install the rest of the Ruby gems by typing the following command:

> gem install thin sinatra rack-accept rest-client sinatra-content-for nokogiri

After you've installed the dependencies for your environment, you're ready to run Deltacloud.

Running Deltacloud

With the dependencies installed in your environment, installing and running Deltacloud for the first time is easy. To install Deltacloud, type the following command:

$ gem install deltacloud-core

When you have finished installing the binaries, start the server by typing the command:

$ deltacloudd -i ec2 -r 192.168.1.2 -p 9988

Specify the driver you're using and optionally the port and host. Specify the host name or IP address as shown if you want to bind to the server from a different computer. Otherwise, by default, the server binds to the local host (127.0.0.1) interface only. In the example above, 192.168.1.2 is the IP address of the laptop on which the server is running.

To see the web application after you start the server, open your browser and go to the URL http://<hostname>:<port>/api, where <hostname> is the host name or IP address and <port> is the port that you supplied on the command line.

While viewing the instances in the web application, you are prompted for a user name and password with HTTP Basic authentication. The values that you enter for your user name and password vary depending on the driver you're using. For Amazon EC2, your user name is your Access Key ID; your password is the secret key. You can read more about how to get those items in Provision an Amazon VM.

You can use the web application as is, without using the RESTful API, to perform any operation the driver supports. Not all drivers support all operations; see Resources for a link to the Deltacloud page, which explains the features available for each driver. It is useful to verify the functionality by trying it in the web application before going on to try it using the RESTful API, so you're not attempting to debug something that's not working.

Now that you have the web application running and have confirmed that it is working, you can proceed to using the RESTful API to create scripts that do the work for you. This article shows you how to use both Bash shell scripting and Windows PowerShell™ scripting for the Linux and Windows operating systems, respectively.


Provision an Amazon VM

To provision a VM using Amazon EC2, you must sign up for an account (see Resources) and have the access key (Access Key ID) and secret access key available. As of the time of this writing, these items are available under My Account/Console > Security Credentials after you have logged in to Amazon Web Services (AWS; see Resources). If you tried using the web application already, you entered these values as the user name and password. You will need them when you write the scripts, as the RESTful API also requires authentication.

Troubleshooting

The first time I tried logging in to Amazon EC2, I used my user name and password for my AWS account instead of the access key-secret key combination. I realized my mistake, but Deltacloud didn't prompt me again for another chance to enter the correct credentials. I could see messages in the verbose output indicating failures to access AWS. Even after several restarts of Deltacloud, I was unable to get the user interface to prompt me again for the passwords. I resolved this issue merely by clearing out the temporary files on my browser.

You need to upload a Secure Shell (SSH) public key to Amazon if you intend to log in to the Amazon VM. You don't need the SSH public key to create an instance, but without it you won't be able to actually log in to the instance and do anything with it. You can successfully create and destroy VM instances without it, however.

Deltacloud did have a command line client, but the current official site lists it as being deprecated. Instead, if you are interested in calling Deltacloud from a command line, you should use a command that simply allows you to call the RESTful API, such as cURL. The examples in this article use the curl command in Bash shell scripts.

To install cURL on a Ubuntu system, type the following command:

$ sudo apt-get install curl

When you have cURL installed, you can write a script that calls your Deltacloud instance to create VM instances on Amazon EC2. See Listing 1 for an example of a Bash shell script for listing out the instances that you have configured.

Listing 1. A Bash shell script that calls cURL
#!/bin/bash

BASE_URL=localhost:9988
ADC_USER=MYACCESSKEY
ADC_PASS=SECRETKEY
URL_OPTS="?format=json"

curl -u ${ADC_USER}:${ADC_PASS} \
--url http://${BASE_URL}/api/instances${URL_OPTS}
echo

The response is in JavaScript Object Notation (JSON) here, because the ?format=json value is appended to the query string of the URL that the curl command calls. You can specify the return format in other ways, as well, such as setting the Accept value in the HTTP header to application/json.

In addition to listing the current instances, you can create new instances with the RESTful API. To create a new instance, you must provide some data. This is the same data that is submitted to the web page when you use it. Listing 2 shows an example of a file with data in it that can be posted to create a new instance.

Listing 2. A file that contains properties
realm_id=us-east-1a
firewalls0=default
metrics=on
hwp_id=t1.micro
image_id=ami-16a3577f
keyname=Lenovo%20Linux%20Mint%20Build
instance_count=1

The script that parses the file and adds each line as a post data parameter (-d) is shown in Listing 3.

Listing 3. The script that creates a new instance
#!/bin/bash

while read LINE; do
    ADC_DATA="${ADC_DATA}&${LINE}"
done < properties.txt

BASE_URL=localhost:9988
ADC_USER=MYACCESSKEY
ADC_PASS=SECRETKEY
URL_OPTS="?format=json"

curl -v -H "Accept: application/json" \
    -X POST -u "${ADC_USER}:${ADC_PASS}" \
    -d "${ADC_DATA}"  http://${BASE_URL}/api/instances

echo

Remember to replace MYACCESSKEY and SECRETKEY with your own values. They should be the same as the user name and password you used to log in to the web application.

The Bash shell script reads the file and builds the command line options for the curl command. The command as submitted looks like that in Listing 4. You can simplify this code by using the command directly in the script instead of building it by using a file. However, the file can be a bit easier to modify, and you can add or subtract different values.

Listing 4. The full curl command as submitted to the RESTful API
$ curl -v -H "Accept: application\json" -X POST \
    -u MYACCESKEY:SECRETKEY \ -d "realm_id=us-east-1a&firewalls0=default\
    &metrics=on&hwp_id=t1.micro&image_id=ami-16a3577f&\
		keyname=Lenovo%20Linux%20Mint%20Build&instance_count=1&" \
        http://localhost:9988/api/instances

After you execute the script, a new instance should appear in both the web application and the Amazon EC2 control panel. To limit additional costs, remember to pay attention to your instances and shut them off when you're finished.


Using Deltacloud with IBM SmartCloud

You must sign up for an account to create an instance of a VM on IBM SmartCloud (see Resources). When you have received an account and can sign in, use your IBM account user name and password as the user name and password when challenged by the Deltacloud web application or services.

The power of Deltacloud is that you can use the same script to list machine instances on different cloud providers with few changes. See Provisioning an Amazon VM for an example of a Bash shell script that uses the curl command to call the RESTful API.

On Windows, a Windows PowerShell script can be the most straightforward method of calling the RESTful API. Listing 5 shows how to use a Windows PowerShell script to list instances of VMs on IBM SmartCloud. I use the Windows PowerShell Integrated Scripting Environment to execute the script so that I can step through it, if needed.

Listing 5. The Windows PowerShell script for listing instances
$username = "MYACCESSKEY"
$password = "SECRETKEY"
$target = "http://localhost:9988/api/instances"

$webRequest = [System.Net.WebRequest]::Create($target)
$webRequest.Accept = "application/json"
$webRequest.ServicePoint.Expect100Continue = $false
$webRequest.Credentials = New-Object System.Net.NetworkCredential 
    -ArgumentList $username, $password 

$webRequest.PreAuthenticate = $true

[System.Net.WebResponse] $resp = $webRequest.GetResponse();
$rs = $resp.GetResponseStream();
[System.IO.StreamReader] $sr = 
    New-Object System.IO.StreamReader -argumentList $rs;
[string] $results = $sr.ReadToEnd();

return $results;

The Windows PowerShell script uses the WebRequest object from the System.Net namespace. It first sets a few variables ($username, $password, $target), and then uses them later. This is just to make the script more readable and maintainable.

The Windows PowerShell script has an example of setting the Accept value for the HTTP headers to application/json to get a JSON response. Also, the authentication credentials are set in the script.


In conclusion

Apache Deltacloud is a web application implemented in Ruby that you can install and run to manage VMs and cloud storage for many different providers. Deltacloud has drivers that attach to different vendors. You can use Deltacloud as a layer of abstraction between you and your cloud vendor for performing operations such as creating instances, checking on their status, and shutting them down. See you in the cloud.

Resources

Learn

Get products and technologies

  • Learn more about and download Apache Deltacloud.
  • Learn more about and find the latest versions of Ruby and RubyGems.
  • To learn more about and find the latest versions of g++, visit GCC Releases.
  • Learn more about and find the latest versions of libxml and libxslt at the XML C parser and toolkit of Gnome and the XSLT C library for GNOME sites, respectively. The libxml2-dev and libxslt-dev packages on Ubuntu systems, which are special packages that contain the header files, are also available on these sites.
  • Download the Ruby installer for Windows to get a self-contained installer for the Windows platform.
  • Download the Ruby DevKit for Windows to be able to install some of the dependencies for Deltacloud.
  • Learn more about and get started with Windows Azure, Microsoft's cloud platform.
  • Learn more about and get started with Windows PowerShell.
  • Evaluate IBM products in the way that suits you best: Download a product trial, try a product online, use a product in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Cloud computing on developerWorks


  • Bluemix Developers Community

    Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

  • Cloud digest

    Complete cloud software, infrastructure, and platform knowledge.

  • DevOps Services

    Software development in the cloud. Register today to create a project.

  • Try SoftLayer Cloud

    Deploy public cloud instances in as few as 5 minutes. Try the SoftLayer public cloud instance for one month.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Cloud computing
ArticleID=852224
ArticleTitle=Use Apache Deltacloud to administer multiple instances with a single API
publish-date=12172012