Infrastructure as a service cloud development
I have uploaded the presentation for this month's Developer Group meeting to the SmartCloud Enterprise asset catalog. The main presentation by Ignacio Silva-Lepe on SmartCloud Application Services (SCAS) and Cloud Services Framework (CSF) is at
The second presentation, by myself, on Using a SOCKS Proxy with VLANs is at
You will need an SCE user account to access these.
In addition to Ignacio's presentation there is a good overview of different methods to deploy complex infrastructure on SmartCloud Enterprise by Dominique Vernier at
AAmies 270002F3W7 Tags:  stingray highavailability riverbed smartcloud smartcloudenterprise cloud iaas 2,769 Views
I have posted this month's slides for the IBM SmartCloud Enterprise Developer's meeting, which is Riverbed Stingray. This is a leading solution for high availability, load balancing, denial of service protection, and other functions. The slides are a documentation asset in the SCE asset catalog. The slides are here:
You will need an SCE user account to access the URL.
There are two very useful Firefox plug-ins for working with the SmartCloud Enterprise REST APIs. Actually, these are useful for working with any REST API's. They are:
I will discuss the User Agent Switcher in this blog entry. This plug-in allows you to change the value for the User Agent HTTP header. It can only be used for GET requests. The plug-in can be installed directly from the Tools | Add-ons menu in Firefox. The project web site is at http://chrispederick.com/work/user-agent-switcher/. After installing the plug-in and restarting Firefox, enter a new user agent profile. Do this by bringing up the Edit User Agents dialog by selecting the Tools | Default user Agent | Edit User Agents menu. This is shown below.
Click the New button and enter values for the fields. Importantly, set the value for User Agent to something that does not contain 'Mozilla' or 'IE'. Browsers are disallowed from invoking the REST APIs with the default User Agent header. This dialog is shown below.
Click OK for both dialogs saving the profile with a name, say 'cloudapi'. Now that you have added the user agent profile select it by clicking on the Tools | Default User Agent | cloudapi. Now you are ready to invoke the REST API. Paste the URL below into your browser's address bar or click on it:
This will fetch the list of virtual machine instances you own. You will be prompted for your user id and password. Then the instances will be displayed in the browser in XML format. Another useful one URL is
Which lists the VLANs assigned to your organization at each data center.
The full set of URLs for the REST API is listed in the SCE REST API Reference. Use the link in the sCE support tab to access that.
Invitation: January 30th IBM SmartCloud Enterprise Developers Group Virtual Meeting OpenStack and CIMI REST API
There are a couple of changes in this meeting, both in time and content, so I am posting it again with the changes. I have also uploaded the slides to the SCE documentation library.
A meeting for SCE customers and users of the REST APIs and related tools to develop solutions on the cloud, previously known as the API Users Group.
Date: Wednesday January 30, 2012 9:00 AM Eastern US
Tong Li and Doug Davis will discuss the CIMI REST Interface that he has created for OpenStack. The Cloud Infrastructure Management Interface (CIMI) is a new standard published in 2012 by the DMTF that will be implemented by multiple clouds. IBM participated in defining the standard and has contributed this as open source to the OpenStack project.
1. News and publications
2. Overview of the OpenStack project
3. Overview of CIMI
4. CIMI implementation for OpenStack
Tong Li is a Senior Software Engineer at IBM where he has been working for over 15 years. He has worked on IBM Websphere Portal Server, LotusLive and is currently working with the IBM Emerging Technologies Division on Cloud computing technologies. Most of his recent work has focused on Cloud storage standard and OpenStack Swift.
Doug Davis is a Senior Technical Staff Member representing IBM on the Distributed Management Task Force Cloud Management Working Group and editor of the CIMI specification.
Updates to the SCE Python driver for Apache LibCloud is available: http://libcloud.apache.org/index.html.
Sonian and a few others have collaborated on a new command line interface for SCE written in Ruby. It can installed and running rapidly and simply and works natively on multiple platforms. Here is the link:
Developing and Hosting Applications on the Cloud, book available at
New article: IBM SmartCloud Enterprise management, Part 3: Clone and modify an asset for customized provisioning
New article: IBM SmartCloud Enterprise management, Part 4: Create images with ICCT
Article published: Workload Migration for the Cloud
If you would like to receive the official calendar invitation for the user group meetings, send an email to email@example.com or firstname.lastname@example.org.
Connecting to remote Linux desktops on is frequently needed in cloud applications. Because the remote machines are on the Internet you should be avoid sending unsecured network traffic to them and also avoid opening insecure services, like VNC, on the Internet.
I have written a technical article in the SCE asset repository that explains how to connect to a remote Linux desktop securing by tunneling VNC traffic over SSH. Besides encrypting the network traffic the technique avoids the need to open the firewall for VNC. The technique is explained for OpenSSH and PuTTY. I presented this at an SCE User Group last year and just updated it to include instructions for tunneling through a firewall to a virtual machine on a private VLAN. The TechNote will be useful to readers who have basic experience with SSH and Linux. The article is here:
AAmies 270002F3W7 Tags:  poster smartcloud iaas cloud rest sce smartcloudenterprise 2,662 Views
I want to add a little to my earlier entry on the use of the Firefox plug-in Poster to call the SmartCloud Enterprise (SCE) REST APIs for managing cloud resources. In the entry Firefox Poster Add-On for Cloud REST APIs I introduced the use of the Poster plug-in but I only gave an example for GET requests. For POST and PUT requests you will need to add a HTTP method body. The SCE REST API expects parameters in a form-encoded body. You can add these easily with Poster. It can also help give you an understanding of the REST API.
Here are the steps to create a storage volume. You need find out the values parameters to add first by doing a GET for volume offerings
2. Add the User-Agent header as per the GET method.
3. Do a GET request to the URL https://www-147.ibm.com/computecloud/enterprise/api/rest/20100331/offerings/storage5. Change to the Parameters tab and enter the parameter names and values for name, offeringID, location, format, and size. Optionally, you can add a storage area ID (storageAreaID) as well.
4. Check the results and note the location ID for the data center that you want to create the storage volume at, the offering ID, and choose a size in GiB
6. Change to the Content to Send tab and click the Body from Parameters button. You should see something like the screen shown below.
The form encoded content to send via HTTP POST
7.Click the POST button. This will send a HTTP POST to the cloud server. After about 20-30 seconds you should see response like shown below.
Poster Response for Create Volume
8. Take a note of the volume ID. You will need it to check the status of the storage volume.
9. You will not be able to use the storage volume until it changes to the UNMOUNTED state. Do a GET request to the same URL to check the status.
AAmies 270002F3W7 Tags:  smartcloudenterprise websphere iaas j2ee cloud smartcloud sce 2,657 Views
WebSphere Application Server (WAS) is the industry leading J2EE server. WAS 8 is built on Java EE 5 and J2SE 6 and supports
The SmartCloud Enterprise WAS virtual machine image is ideal for using WebSphere in production or leaning to use it, perhaps studying for Websphere certification. To start up WAS in SCE UI follow these steps:
(1) go to the Control Panel and click on the 'Add Instance' button
(2) In Step 1 of 4 select a data center and select the image 'IBM WebSphere Application Server 8 Standalone Node and click next
(3) Choose a compute size, as shown below
(4) Enter the parameters for configuring WAS, including administrative ID and password and click Next
(5) Click Next again, agree to the Terms and Conditions and click Submit.
(6) Check the Control Panel for the virtual machine instance to be provisioned. It will take several minutes.
(7) After it has been provisioned take a note of the IP address. You can log in to the operating system with an SSH client.
The WebSphere 8 image is build on Red Hat Enterprise Linux. If you have chosen to configure the IHS server, as in step 4 above, you will now be able to point your web browser to the IP address or host name of the virtual machine and will see the IHS welcome page. To connect to the WebSphere Integrated Solutions Console enter the address below into your browser:
http://<fully_qualified_hostname or IP>:9060/ibm/console
and enter the user ID and password that you set in step 4 above. You should see a screen like shown below.
See the book Developing and Hosting Applications on the Cloud for more details on using SmartCloud Enterprise and related offerings from IBM.
Note that you will need an SCE account to try this out. You can sign up online at https://www.ibm.com/cloud/enterprise. Please note that the image catalog in SCE is updated regularly.
Developing and Hosting Applications on the Cloud book: http://www.amazon.com/dp/B008FP2G5O
IBM SmartCloud Enterprise: https://www.ibm.com/cloud/enterprise/dashboard
WebSphere Application Server Network Deployment V8.0, Core Administration certification: http://www-03.ibm.com/certify/tests/ovr317.shtml
IBM WebSphere Application Server V8 Concepts, Planning, and Design Guide redbook: http://www.redbooks.ibm.com/abstracts/sg247957.html?Open
WebSphere Application Server V8: Administration and Configuration Guide redbook: http://www.redbooks.ibm.com/redpieces/abstracts/sg247971.html?Open
WebSphere Application Server information center: http://pic.dhe.ibm.com/infocenter/wasinfo/v8r0/index.jsp
The ideas expressed in this blog are my own and do not represent IBM.
It is a frequent requirement to copy large files to the cloud. In particular, OVF files for virtual machine images are very large. The problem with copying very large files, over 1G, with SCP is that the SSH pipe may break. This can be frustrating in the middle of a large copy operation. rsync can provide a solution to this. Unzip the large files on the source machine into a directory and run rysnc. It is a tool that keeps directories synchronized and sends the compressed changes over SSH. So if it does not complete successfully you can run it again multiple times and only the differences will be copied until, finally, both sets of directories will be in sync.
See the last post for more details of using rsync on the cloud.
AAmies 270002F3W7 Tags:  sce virtualization iaas smartcloudenterprise cloud smartcloud qemu 2,547 Views
Presentation: Recover virtual machine instances using QEMU
A meeting for SCE customers and users of the REST APIs and related tools to develop solutions on the cloud. Previously known as the API Users Group.
Date: Wednesday November 28, 2012 10:00 AM Eastern US
Agenda: Recover virtual machine instances using QEMU
Speaker: Mihai Criveti, Image development specialist and cloud architect, IBM
Presentation materials: Location will be posted on this blog
Meeting information: email aamies(at)cn.ibm.com for call-in, web conference details, and a calendar invitation.
The IBM SmartCloud Enterprise Developers Group (previously API User Group) is a technical community composed of individuals interested in the application programming interfaces (APIs) of the IBM SmartCloud Enterprise offering. The group includes customers, IBM Business Partners, independent software vendors, and IBM employees who share a common interest in understanding the APIs of the IBM SmartCloud Enterprise offering and how they can be used to automate processes and build solutions that integrate with IBM SmartCloud Enterprise.
The IBM SmartCloud Developers Group is hosted by Alex Amies. Alex is a Senior Software Engineer in the GTS Development Lab team. He has been with IBM since 2002. His current assignment is Architect with the design team for the IBM SmartCloud Enterprise public cloud offering at ibm.com/cloud/enterprise. Alex has been working in the China Development Lab in Beijing since 2009. Prior to 2009 Alex was product architect for IBM Tivoli Identity Manager, based in Costa Mesa, California.
There a similarities and differences in working with Windows and Linux virtual machines on SmartCloud Enterprise. To provisioning a Windows virtual machine follow the same provisioning wizard in the SCE web portal that you do for Linux virtual machines. However, after you have created it you will connect to it with Remote Desktop Protocol (RDP) instead of SSH, as you would with Linux. RDP is a secure, high performance protocol. To open a remote desktop to your Windows virtual machine select Remote Desktop Connection from the Start menu under Accessories. The Remote Desktop Connection dialog is shown below.
To transfer files to the virtual machine, map a drive using the Remote Desktop Connection dialog before you connect. You can set this using the Options drop down in the dialog and then changing to the Local Resources tab and mapping a drive. The dialog is shown below.
More details are given in the Working with your Windows instances section of the SCE User Guide.
Start-up logs are not available for Windows images and image import / copy is not supported for Windows as it is for Linux. There are also a some additional steps and configuration settings to be mindful of when working with Windows. This can make it harder to debug when you are customizing a Windows image. Therefore, it is good practice to keep running instances for important images alive until you can verify that captured images are able to be used for provisioning new virtual machines. Be careful to follow the steps listed in the section Working with your Windows instances of the SCE User Guide and chapters 3 and 4 of the Customizing Image and Software Bundles Reference.
Lisa Seacat DeLuca summarizes the differences and similarities of working with Windows virtual machines and describes techniques for automation in her developerWorks article Customize SmartCloud Enterprise Windows-based images:
Windows PowerShell can be a useful scripting tool when customizing Windows images. There is a extract-parameters.ps1 script provided in SCE Windows images in C:\Windows\System32 folder to extract provisioning parameters from the parameters.xml file.
This article has just been published on IBM developerWorks. Thanks to the lead author Alexei Karve and other authors. The article is at this URL:
A software bundle is a collection of software installation files, configuration files, and metadata that can be deployed on a virtual machine instance. In this final article, the authors describe the three mechanisms to create images using the IBM Image Construction and Composition Tool (ICCT) to capture a customized image, and the mechanism to manually add a dependency between an image and a software bundle by updating the semantic topology file. The ICCT performs bookkeeping tasks and provides a step-by-step process and a graphical interface to conveniently create software bundles, customize virtual machine instances, and capture images. This series shows how to use ICCT to create a software bundle, then register and maintain it on IBM SmartCloud Enterprise.
This article describes how to integrate set-up and tear-down of virtual environments as part of the build process with Apache Maven. Thanks to my co-authos Perry (Pan Xia Zou) and Shirley (Yi Shuai Wang) for their help in writing it. It now has 43088 views on developerWorks. Here is the link to the article:
Summary: A recent trend has been to build a strong connection between cloud application and service development and operations; in particular, this trend is leading to a tighter, more efficient integration of application life cycle management (ALM) tools with cloud computing. In this article, the authors will show you how to use the open source Apache Maven build management tool to automate build and deployment projects on IBM® SmartCloud Enterprise. They will also demonstrate how to integrate the management of virtual machines on the cloud into the build and deployment life cycle by developing an Apache Maven plug-in that looks up and creates virtual machines that run a J2EE application server on the cloud. You'll also discover best practices for development and deployment on the cloud and how to use IBM Cloud API's and Maven to implement these practices.
This paper was published on IBM developerWorks today. The paper describes concepts and tools for networking on the cloud. Although the we make specific reference to IBM SmartCloud Enterprise the concepts and techniques are applicable to any cloud. Thanks to my co-authors Chun Feng, Guang Cai, and Mihai for their help on writing this paper. The paper is here:
Summary: Join the authors as they describe important concepts of networking for IBM® SmartCloud Enterprise, including virtual local area networks (VLANs), virtual private networks (VPNs), and the various protocol layers. They also explain how to use tools such as OpenSSH, OpenVPN, and proxy servers to set up different network topologies and solve connectivity problems from within the context of common organizational scenarios and security issues and practices.
I also have a slide deck with a summary of the material that I plan to use for the SmartCloud Enterprise Developers group next week:
In this post I will highlight some experience with Rational tools on IBM SmartCloud Enterprise (SCE). Rational development tools support cloud in several different ways: by being available as pre-built virtual machine images for use on the cloud; by providing a way to manage cloud resources, and by providing tools that can directly support cloud infrastructures. The use of Rational tools work on SCE are outlined in this presentation:
that I gave with Scott Peddle and Harm Sluiman at Innovate last year. The deck outlines business scenarios for use of Rational tools in cloud projects, describes the different Rational tools available in the SCE catalog, and describes the use of Rational Asset Manager in SCE. The Rational tools on SCE include Rational Asset Manager, Rational Application Developer, Rational Quality Manager, Testing as a Service, Rational Software Architect, Rational Build Forge, and Rational Team Concert. The ability to provision and manage J2EE servers on SCE directly from Rational Application Developer and Rational Software Architect is particularly useful. There is a deep dive on the use of Rational Asset Manager to support cloud-based asset development, especially development of virtual machine images.
Also, just a reminder that the IBM SmartCloud Enterprise promotion is still ongoing and allows new customers to test drive the cloud for free for a limited period. Details here:
Thanks to everybody who attended the presentation. I have uploaded the presentation to slideshare.net. I think that it will be useful to anybody using the cloud, not just IBM SmartCloud Enterprise
The topic for the November meeting will be Recovering inaccessible instances using QEMU.