The Simple Cloud API

Writing portable, interoperable applications for the cloud

The recently announced Simple Cloud API provides a common API to a variety of cloud services. A collaborative effort by Zend, GoGrid, IBM®, Microsoft®, Nirvanix and Rackspace, the API allows you to write portable code that can interoperate with multiple cloud vendors. Best of all, the API allows you to use services specific to a particular vendor as necessary.

Share:

Doug Tidwell, Cloud Computing Evangelist, Software Group Strategy, EMC

Doug TidwellDoug Tidwell is a senior software engineer in IBM’s Emerging Technology group. He was a speaker at the first XML conference in 1997 and has been working with markup languages, Web services and SOA technologies for many years. His job as a technology evangelist is to explain the standards and technologies behind cloud computing and to help customers integrate them into their overall business architectures and strategies. He is the author of many articles here at developerWorks and is the author of O’Reilly’s book on XSLT (ISBN 0-596-52721-7), a copy of which makes a thoughtful gift for all occasions. He lives in Chapel Hill, N.C., with his wife, daughter, and dog.


developerWorks Contributing author
        level

20 October 2009

Also available in Japanese

The flexibility and economic benefits of cloud computing have generated a tremendous amount of interest. As developers work with this technology, an obvious concern is vendor lock-in. Writing an application that makes the most of cloud computing is great. But what if that application locks you in to a single vendor? The Simple Cloud API is an effort by multiple cloud vendors to create a single API that works with cloud services from multiple providers (see Resources). This article is a high-level overview of the API and its goals.

The state of the art

Cloud computing promises to change IT as dramatically as anything since the rise of the Web. If you're a developer or architect, you should at least be evaluating cloud computing if you're not using it already. These are exciting times, but the usual caveats apply. As with any new technology, you should understand cloud computing's strengths and weaknesses and use it only where it's appropriate.

One of the biggest risks with any new technology is building critical applications that lock you in to a particular vendor. Cloud computing can have significant economic benefits, but if using the cloud puts you at the mercy of a vendor's pricing strategy, that's a major concern. Even if you're perfectly happy with your cloud provider, your partners, customers and suppliers will likely use different cloud providers. It's vital that your applications be able to work with as many providers and services as possible.

The Simple Cloud API is designed to provide a single, simple, interoperable API for multiple cloud services and multiple cloud providers.


Levels of APIs

There are several ways of writing code to work with cloud services. To put the Simple Cloud API in context, we'll look at APIs at four levels:

  • The wire
  • Language-specific toolkits
  • Service-specific toolkits
  • Service-neutral toolkits

We'll consider these in detail next.

The wire

At this level, you are concerned with the actual bytes that go across the connection from the application to the cloud. For a SOAP service, the application has to build the <SOAP:Envelope> with the appropriate contents and headers. For a REST service, the application has to create the correct HTTP headers and build a URL that contains the appropriate parameters. For example, a REST request to the Nirvanix Internet Media File System (IMFS) to list all the items in the directory /dougtidwell might look like Listing 1.

Listing 1. Wire format of a sample REST request
GET /ws/IMFS/ListFolder.ashx?sessionToken=8da051b0-a60f-4c22-a8e0-d9380edafa6f&...HTTP/1.1
Host: services.nirvanix.com
Date: Wed, 20 Oct  2009 12:00:00 GMT

This is a request for a list of all the folders in the directory /dougtidwell. The request includes authentication information (the sessionToken parameter above), along with parameters like path name, page number, and page size. The complete URL in Listing 1 would include FolderPath=/dougtidwell&PageNumber=1&PageSize=5.

Language-specific toolkits

A language-specific toolkit provides convenience classes to create the SOAP and REST data structures. As a developer, you are still focused on the data structures that pass between your application and the cloud, you don't have to create the data structures directly. For example, the Zend_Soap library contains classes to simplify invoking a SOAP service. Invoking a SOAP service looks something like Listing 2.

Listing 2. Calling a SOAP service with Zend_Soap_Client
$params = array(..., 'FolderPath' => '/dougtidwell', 'PageNumber' => 1, ...);
$soapClient->call('listFolder', $params, $namespace);

The powerful curl libraries are a useful way to work with REST services. A request to retrieve data via REST might look like Listing 3.

Listing 3. Calling a REST service with curl
$curl_proc = curl_init('http://services.nirvanix.com/ws/IMFS/ListFolder.ashx');
$curl_post_data = array('sessionToken' => '8da051b0-a60f-4c22-a8e0...', 
                        'folderPath'   => '/dougtidwell',
                        'pageNumber'   => 1, 
                        'pageSize'     => 5);
curl_setopt($curl_proc, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl_proc, CURLOPT_POST, true);
curl_setopt($curl_proc, CURLOPT_POSTFIELDS, $curl_post_data);
$response = curl_exec($curl_proc);

In both of these examples, the application isn't concerned with parsing XML or checking HTTP return codes or similar issues; the toolkits handle most of the details. On the other hand, the application is not focused on business objects.

Service-specific toolkits

At this level, you are focused on business objects. Using higher-level objects that encapsulate cloud services means you no longer have to think about the format of the data on the wire. Working at this level, you have no idea whether the underlying service is SOAP or REST; you simply invoke the service. Two examples of service-specific toolkits are the Zend Framework classes for Nirvanix and Amazon S3. Following is the method for listing all of the items in a Nirvanix directory.

Listing 4. Listing all the items in a Nirvanix directory
$auth = array('username' => 'your-username', 
              'password' => 'your-password', 
              'appKey'   => 'your-appkey');
$nirvanix = new Zend_Service_Nirvanix($auth);
$imfs = $nirvanix->getService('IMFS');

$args = array('folderPath' => '/dougtidwell', 
              'pageNumber' => 1, 
              'pageSize'   => 5);
$stuff = $imfs->ListFolder($args);

Nirvanix provides a number of services. The example here uses IMFS. Calling the getService() method returns an object you can use to interact with a particular service. Passing parameters to the ListFolder method returns an array of the items in the folder.

In contrast to Nirvanix's directory structure, Amazon's S3 uses buckets. Buckets cannot contain other buckets, so the hierarchical structure of the Nirvanix file system isn't supported. The simpler data model of S3 is reflected in the code for listing all of the items in an S3 bucket.

Listing 5. Listing all the items in an S3 bucket
$s3 = new Zend_Service_Amazon_S3($accessKey, $secretKey);
$stuff = $s3->getObjectsByBucket($bucketName);

This sample code retrieves the meta information about all the items in a particular bucket. The array $stuff has metadata about each of those items, such as the item's name, size, content type, and timestamp.

Service-neutral toolkits

Service-specific toolkits improve developer productivity. However, a service-specific API locks you in to a particular service and provider. The goal of the Simple Cloud API is to provide a set of high-level classes that let you write one piece of code that can work with multiple services and providers. In looking at the Nirvanix and S3 classes, listing the contents of a directory or bucket required the ListFolder() and getObjectsByBucket() methods, respectively. In the Simple Cloud API, the ListItems() method does the job for both services.


The Simple Cloud API

The Simple Cloud API is designed for interoperable code. Operations defined in the Simple Cloud API are supported by many cloud services. The ultimate goal is that the code written to work with one cloud service should work with all similar cloud services. The PHP implementation of the Simple Cloud API uses the Factory and Adapter design patterns. To work with a particular cloud service, you call the appropriate factory method (such as Zend_Cloud_Storage_Factory for file storage) with an array of configuration parameters. The class returned by the factory method is an adapter to a particular cloud service. The adapter maps the Simple Cloud API calls to the service-specific calls required by each cloud provider.

The Simple Cloud API is defined for three types of cloud services:

  • File storage
  • Document storage
  • Simple queues

File storage refers to traditional cloud storage systems like S3 and Nirvanix. With a file storage service, you're responsible for knowing what data you have stored in the cloud. You can get a list of directories/buckets and a list of the files in each, but it's up to you to keep track of what each file represents. Typical methods in this API are fetchItem(), listItems(), deleteItem(), and fetchMetadata().

Document storage includes more-structured systems, such as Amazon's SimpleDB. Unlike simple file storage, document storage provides query facilities to help you find information. In some cases, the underlying service will be a relational database with schema support. In other cases, it will be a much simpler type of service. Typical methods in this API are listCollections(), listDocuments(), insertDocument(), and query().

Simple queues are queueing systems like Amazon's Simple Queue Service. Typical methods for the simple queue API include sendMessage(), listQueues(), and peekMessage().


A simple sample

Note: The Simple Cloud API is still a work in progress, and details are subject to change. The sample application here works with the adapters for Amazon S3 and Nirvanix IMFS as of the date of this article's publication.

To illustrate the interoperability provided by the Simple Cloud API, we'll build a simple PHP page that works with Amazon S3 and Nirvanix IMFS. The code begins by using the Zend_Config_Ini class to create an array of credentials from an .ini file. (The factory classes in the Simple Cloud API require an array of credentials as a parameter.) The two configuration files used in this sample are s3.ini and nirvanix.ini. If the name of a configuration file is not passed as a parameter, the code uses s3.ini by default.

Listing 6. Creating an array of credentials
require_once 'Zend/Cloud/Storage/Factory.php';
require_once 'Zend/Http/Client/Adapter/Socket.php';
require_once 'Zend/Config/Ini.php';

if (array_key_exists('configfile', $_POST))
  $configFile = $_POST['configfile'];
else
  $configFile = '../conf/s3.ini';

$credentials = new Zend_Config_Ini($configFile);

The sample configuration file for an Amazon S3 adapter looks like Listing 7.

Listing 7. Configuration file for Amazon S3
http_adapter = "Zend_Http_Client_Adapter_Socket"
storage_adapter = "Zend_Cloud_Storage_Adapter_S3"
bucket_name = "developerworks"
aws_accesskey = "12345678901234567890"
aws_secretkey = "ABcdEFghIJklMNopQRstUVwxYZ01234567890abc"

The storage_adapter value is the name of the PHP class that implements the StorageService interface. For S3, the class is Zend_Cloud_Storage_Adapter_S3. A Nirvanix configuration file requires an application name (currently represented by the parameter auth_accesskey) and a directory name, in addition to a username and password.

Listing 8. Configuration file for Nirvanix IMFS
storage_adapter = "Zend_Cloud_Storage_Adapter_Nirvanix"
auth_accesskey = "12345678-90ab-cdef-1234-567890abcdef"
auth_username = "nobody"
auth_password = "xxxxxxxx"
remote_directory = "/dougtidwell"

The configuration file has the details that the Zend_Cloud_Storage_Factory class needs to create the appropriate adapter. Following is the code to get the list of items from the cloud storage service.

Listing 9. Using the factory class
<ul style='font-size: 125%; font-family: monospace; font-weight: bold;'>
  <?php
  $stuff = Zend_Cloud_Storage_Factory::getAdapter($credentials)->listItems();
  foreach ($stuff as $nextItem) {
    echo "<li>".$nextItem."</li>";
  }
  ?>
  </ul>

Using the default configuration file means that the information from s3.ini is used. That displays the contents of the S3 bucket named in the configuration file.

Figure 1. Bucket or directory listing with default configuration file
Graphic representation of bucket or directory listing with default configuration file

The bottom of the screen has two buttons that let you reload the page with a particular configuration file. Clicking on Create adapter using nirvanix.ini runs the exact same code, but with different configuration parameters. The results display the contents of the Nirvanix directory named in the configuration file.

Figure 2. Bucket or directory listing with a different configuration file
Graphic representation of bucket or directory listing with different configuration file

These two sets of results were generated by the exact same line of code. The configuration file changed, but the application itself did not.


Summary

The Simple Cloud API is a cooperative effort of several major cloud vendors to create a single, simple, interoperable API that works with many cloud services and providers. Work is under way to add support for more cloud services and cloud providers, and implementations of the API in languages other than PHP are in progress, as well. Cloud computing won't reach its full potential without openness and flexibility. The Simple Cloud API is an important tool for keeping the cloud open and your applications flexible.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Open source on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Open source, Cloud computing
ArticleID=436527
ArticleTitle=The Simple Cloud API
publish-date=10202009