Infrastructure as a service cloud development
I have just published the article Creating a DNS Service on the Cloud. In the article we discuss how to configure and use a DNS server for management of virtual machines on the cloud. This is described in the context of a cloud-based virtual local area network and scripting to automate network configuration of virtual machines. The procedures in this article were tested with IBM SmartCloud Enterprise (SCE). The principles we discuss apply to other Infrastructure as a Service (IaaS) clouds as well. However, no warranty is given that the tools and techniques described here will work.
Article link: http://www.informit.com/articles/article.aspx?p=2015293
AAmies 270002F3W7 Tags:  virtualization sce iaas smartcloudenterprise smartcloud cloud qemu 3,378 Views
Presentation: Recover virtual machine instances using QEMU
A meeting for SCE customers and users of the REST APIs and related tools to develop solutions on the cloud. Previously known as the API Users Group.
Date: Wednesday November 28, 2012 10:00 AM Eastern US
Agenda: Recover virtual machine instances using QEMU
Speaker: Mihai Criveti, Image development specialist and cloud architect, IBM
Presentation materials: Location will be posted on this blog
Meeting information: email aamies(at)cn.ibm.com for call-in, web conference details, and a calendar invitation.
The IBM SmartCloud Enterprise Developers Group (previously API User Group) is a technical community composed of individuals interested in the application programming interfaces (APIs) of the IBM SmartCloud Enterprise offering. The group includes customers, IBM Business Partners, independent software vendors, and IBM employees who share a common interest in understanding the APIs of the IBM SmartCloud Enterprise offering and how they can be used to automate processes and build solutions that integrate with IBM SmartCloud Enterprise.
The IBM SmartCloud Developers Group is hosted by Alex Amies. Alex is a Senior Software Engineer in the GTS Development Lab team. He has been with IBM since 2002. His current assignment is Architect with the design team for the IBM SmartCloud Enterprise public cloud offering at ibm.com/cloud/enterprise. Alex has been working in the China Development Lab in Beijing since 2009. Prior to 2009 Alex was product architect for IBM Tivoli Identity Manager, based in Costa Mesa, California.
AAmies 270002F3W7 Tags:  smartcloudenterprise websphere iaas j2ee cloud smartcloud sce 3,363 Views
WebSphere Application Server (WAS) is the industry leading J2EE server. WAS 8 is built on Java EE 5 and J2SE 6 and supports
The SmartCloud Enterprise WAS virtual machine image is ideal for using WebSphere in production or leaning to use it, perhaps studying for Websphere certification. To start up WAS in SCE UI follow these steps:
(1) go to the Control Panel and click on the 'Add Instance' button
(2) In Step 1 of 4 select a data center and select the image 'IBM WebSphere Application Server 8 Standalone Node and click next
(3) Choose a compute size, as shown below
(4) Enter the parameters for configuring WAS, including administrative ID and password and click Next
(5) Click Next again, agree to the Terms and Conditions and click Submit.
(6) Check the Control Panel for the virtual machine instance to be provisioned. It will take several minutes.
(7) After it has been provisioned take a note of the IP address. You can log in to the operating system with an SSH client.
The WebSphere 8 image is build on Red Hat Enterprise Linux. If you have chosen to configure the IHS server, as in step 4 above, you will now be able to point your web browser to the IP address or host name of the virtual machine and will see the IHS welcome page. To connect to the WebSphere Integrated Solutions Console enter the address below into your browser:
http://<fully_qualified_hostname or IP>:9060/ibm/console
and enter the user ID and password that you set in step 4 above. You should see a screen like shown below.
See the book Developing and Hosting Applications on the Cloud for more details on using SmartCloud Enterprise and related offerings from IBM.
Note that you will need an SCE account to try this out. You can sign up online at https://www.ibm.com/cloud/enterprise. Please note that the image catalog in SCE is updated regularly.
Developing and Hosting Applications on the Cloud book: http://www.amazon.com/dp/B008FP2G5O
IBM SmartCloud Enterprise: https://www.ibm.com/cloud/enterprise/dashboard
WebSphere Application Server Network Deployment V8.0, Core Administration certification: http://www-03.ibm.com/certify/tests/ovr317.shtml
IBM WebSphere Application Server V8 Concepts, Planning, and Design Guide redbook: http://www.redbooks.ibm.com/abstracts/sg247957.html?Open
WebSphere Application Server V8: Administration and Configuration Guide redbook: http://www.redbooks.ibm.com/redpieces/abstracts/sg247971.html?Open
WebSphere Application Server information center: http://pic.dhe.ibm.com/infocenter/wasinfo/v8r0/index.jsp
The ideas expressed in this blog are my own and do not represent IBM.
AAmies 270002F3W7 Tags:  riverbed highavailability stingray smartcloudenterprise smartcloud cloud iaas 3,354 Views
I have posted this month's slides for the IBM SmartCloud Enterprise Developer's meeting, which is Riverbed Stingray. This is a leading solution for high availability, load balancing, denial of service protection, and other functions. The slides are a documentation asset in the SCE asset catalog. The slides are here:
You will need an SCE user account to access the URL.
You can view the Javadoc for the SmartCloud Enterprise (SCE) Java API directly at this URL:
You need an SCE user account to access that URL. Sign up for the trial if you don't have an account already!
In addition, the asset download for the Java API for includes a Javadoc zip bundle that is handy to use in your IDE. For example, in Rational Software Architect or Eclipse you can mouse over the API's to view the Javadoc, as shown below.
Notice that it includes the Javadoc zip, as shown in the screenshot below.
Download all the files by clicking the Download this Asset button.
2. Enable Javadoc in the Java Build Path tab of the Project Properites dialog. Open up the entry for DeveloperCloud_API_Client-2.1.jar, as shown below.
It is a frequent requirement to copy large files to the cloud. In particular, OVF files for virtual machine images are very large. The problem with copying very large files, over 1G, with SCP is that the SSH pipe may break. This can be frustrating in the middle of a large copy operation. rsync can provide a solution to this. Unzip the large files on the source machine into a directory and run rysnc. It is a tool that keeps directories synchronized and sends the compressed changes over SSH. So if it does not complete successfully you can run it again multiple times and only the differences will be copied until, finally, both sets of directories will be in sync.
See the last post for more details of using rsync on the cloud.
AAmies 270002F3W7 Tags:  iaas smartcloud poster rest cloud sce smartcloudenterprise 3,265 Views
I want to add a little to my earlier entry on the use of the Firefox plug-in Poster to call the SmartCloud Enterprise (SCE) REST APIs for managing cloud resources. In the entry Firefox Poster Add-On for Cloud REST APIs I introduced the use of the Poster plug-in but I only gave an example for GET requests. For POST and PUT requests you will need to add a HTTP method body. The SCE REST API expects parameters in a form-encoded body. You can add these easily with Poster. It can also help give you an understanding of the REST API.
Here are the steps to create a storage volume. You need find out the values parameters to add first by doing a GET for volume offerings
2. Add the User-Agent header as per the GET method.
3. Do a GET request to the URL https://www-147.ibm.com/computecloud/enterprise/api/rest/20100331/offerings/storage5. Change to the Parameters tab and enter the parameter names and values for name, offeringID, location, format, and size. Optionally, you can add a storage area ID (storageAreaID) as well.
4. Check the results and note the location ID for the data center that you want to create the storage volume at, the offering ID, and choose a size in GiB
6. Change to the Content to Send tab and click the Body from Parameters button. You should see something like the screen shown below.
The form encoded content to send via HTTP POST
7.Click the POST button. This will send a HTTP POST to the cloud server. After about 20-30 seconds you should see response like shown below.
Poster Response for Create Volume
8. Take a note of the volume ID. You will need it to check the status of the storage volume.
9. You will not be able to use the storage volume until it changes to the UNMOUNTED state. Do a GET request to the same URL to check the status.
Connecting to remote Linux desktops on is frequently needed in cloud applications. Because the remote machines are on the Internet you should be avoid sending unsecured network traffic to them and also avoid opening insecure services, like VNC, on the Internet.
I have written a technical article in the SCE asset repository that explains how to connect to a remote Linux desktop securing by tunneling VNC traffic over SSH. Besides encrypting the network traffic the technique avoids the need to open the firewall for VNC. The technique is explained for OpenSSH and PuTTY. I presented this at an SCE User Group last year and just updated it to include instructions for tunneling through a firewall to a virtual machine on a private VLAN. The TechNote will be useful to readers who have basic experience with SSH and Linux. The article is here:
I have uploaded the presentation for this month's Developer Group meeting to the SmartCloud Enterprise asset catalog. The main presentation by Ignacio Silva-Lepe on SmartCloud Application Services (SCAS) and Cloud Services Framework (CSF) is at
The second presentation, by myself, on Using a SOCKS Proxy with VLANs is at
You will need an SCE user account to access these.
In addition to Ignacio's presentation there is a good overview of different methods to deploy complex infrastructure on SmartCloud Enterprise by Dominique Vernier at
This article describes how to integrate set-up and tear-down of virtual environments as part of the build process with Apache Maven. Thanks to my co-authos Perry (Pan Xia Zou) and Shirley (Yi Shuai Wang) for their help in writing it. It now has 43088 views on developerWorks. Here is the link to the article:
Summary: A recent trend has been to build a strong connection between cloud application and service development and operations; in particular, this trend is leading to a tighter, more efficient integration of application life cycle management (ALM) tools with cloud computing. In this article, the authors will show you how to use the open source Apache Maven build management tool to automate build and deployment projects on IBM® SmartCloud Enterprise. They will also demonstrate how to integrate the management of virtual machines on the cloud into the build and deployment life cycle by developing an Apache Maven plug-in that looks up and creates virtual machines that run a J2EE application server on the cloud. You'll also discover best practices for development and deployment on the cloud and how to use IBM Cloud API's and Maven to implement these practices.
There is a WebSphere 8.5 image available in the SCE image catalog. To find it search on the text 'IBM WebSphere Application Server Version 8.5' in the asset catalog to find the different versions and data centers. The image ID will be displayed along with other topology details. It is available in both in RHEL and SUSE flavors. I used the IBM WebSphere Application Server Version 8.5(32-bit) for Red Hat Enterprise Linux 6.2 (32-bit), which includes the WebSphere Customization Tools, Web 2.0, and Mobile Toolkit. You can create an instance of the image using the self service console, as shown below.
The next screen prompts you for several parameters, as shown below.
If you prefer to use a the command line then you can do so with the following commands. I created the server ona VLAN in the Singapore data center. Firstly, to find the VLAN ID use the command
> ic-describe-vlans.cmd -u <user id> -g <password file> -w <passphrase>
I got the image ID from searching the catalog. Next to find the supported sizes used the describe-image command, with the image id being the -k parameter
> ic-describe-image.cmd -u <user id> -g <password file> -w <passphrase> -k 20056229
Executing action: DescribeImage ...
Name: IBM WebSphere Application Server Version 8.5 32b - BYOL
. . .
InstanceType ID: COP32.1/2048/60
Label: Copper 32 bit
Detail: Copper - 32 bit (vCPU: 1, RAM: 2 GiB, Disk: 60 GiB)
. . .
Check parameters.xml for optional parameters. To provision a VM in Singapore use the command below.
where -n specifies the name to use for the new instance, -x is the VLAN ID, -k is the image ID, -L is the data center ID, and -m is the custom parameters (from paramaters.xml).After the instance has been provisioned you log onto the virtual machine with your SSH client (eg, Putty) to see how it is set up. Use the manageprofiles.sh to check the profiles configured, as shown below.
If you do not like the profiles installed you can add your own using either the manageprofiles command or the WebSphere Customization Toolbox (WCT). WCT is tool with a graphical user interface. To create a profile using the manageprofiles command first Change to virtuser
To create a new profile use the command below
INSTCONFSUCCESS: Success: Profile AppSrv01 now exists. Please consult /opt/IBM/WebSphere/AppServer/profiles/AppSrv01/logs/AboutThisProfile.txt for more information about this profile.
To use WCT you need access to the desktop. One of the advantages of provisioning into a VLAN and accessing with a VPN is that there is no need for tunnelling for VNC. Open firewall if on VPN or do tunnelling on port 5901 if on Internet. Change back to idcuser to edit the firewall configuration file.
$ sudo vi /etc/sysconfig/iptables
Add a row to open port 5901, save the file, and restart the firewall.
$ sudo /sbin/service iptables restart
Start the VNC server with the command
Use the VNC client to connect from your workstation to the virtual machine instance desktop. Change to user virtuser again and execute the command below to launch WCT.
You should see something like the screenshot below
Click on 'Launch the Selected Tool' to start the Profile Management Tool. You can add a new profile using the new profile wizard, as shown below.
There are more details in book Developing and Hosting Applications on the Cloud
There a similarities and differences in working with Windows and Linux virtual machines on SmartCloud Enterprise. To provisioning a Windows virtual machine follow the same provisioning wizard in the SCE web portal that you do for Linux virtual machines. However, after you have created it you will connect to it with Remote Desktop Protocol (RDP) instead of SSH, as you would with Linux. RDP is a secure, high performance protocol. To open a remote desktop to your Windows virtual machine select Remote Desktop Connection from the Start menu under Accessories. The Remote Desktop Connection dialog is shown below.
To transfer files to the virtual machine, map a drive using the Remote Desktop Connection dialog before you connect. You can set this using the Options drop down in the dialog and then changing to the Local Resources tab and mapping a drive. The dialog is shown below.
More details are given in the Working with your Windows instances section of the SCE User Guide.
Start-up logs are not available for Windows images and image import / copy is not supported for Windows as it is for Linux. There are also a some additional steps and configuration settings to be mindful of when working with Windows. This can make it harder to debug when you are customizing a Windows image. Therefore, it is good practice to keep running instances for important images alive until you can verify that captured images are able to be used for provisioning new virtual machines. Be careful to follow the steps listed in the section Working with your Windows instances of the SCE User Guide and chapters 3 and 4 of the Customizing Image and Software Bundles Reference.
Lisa Seacat DeLuca summarizes the differences and similarities of working with Windows virtual machines and describes techniques for automation in her developerWorks article Customize SmartCloud Enterprise Windows-based images:
Windows PowerShell can be a useful scripting tool when customizing Windows images. There is a extract-parameters.ps1 script provided in SCE Windows images in C:\Windows\System32 folder to extract provisioning parameters from the parameters.xml file.
This article has just been published on IBM developerWorks. Thanks to the lead author Alexei Karve and other authors. The article is at this URL:
A software bundle is a collection of software installation files, configuration files, and metadata that can be deployed on a virtual machine instance. In this final article, the authors describe the three mechanisms to create images using the IBM Image Construction and Composition Tool (ICCT) to capture a customized image, and the mechanism to manually add a dependency between an image and a software bundle by updating the semantic topology file. The ICCT performs bookkeeping tasks and provides a step-by-step process and a graphical interface to conveniently create software bundles, customize virtual machine instances, and capture images. This series shows how to use ICCT to create a software bundle, then register and maintain it on IBM SmartCloud Enterprise.
Thanks to everybody who attended the presentation. I have uploaded the presentation to slideshare.net. I think that it will be useful to anybody using the cloud, not just IBM SmartCloud Enterprise
The topic for the November meeting will be Recovering inaccessible instances using QEMU.
The Monitored Directory Deployment feature in WebSphere Application Server 8.5 is very convenient to use for deploying or updating applications in the cloud. Once set up, applications can be updated simply by copying to the monitored directory by secure copy (SCP). The advantages over using the WebSphere Integrated Solutions Console (ISC) are that you do not need open ports in addition to port 22 and the update can be easily scripted with generic file manipulation commands. To enable Monitored Directory Deployment in ISC go to Global Deployment Settings | Monitored Directory Deployment, as shown below.
Configuring Monitored Directory Deploy in the WebSphere ISC
> scp -i <key_file> <local_source_file> idcuser@<vm_host_name>:<target_file>
The command format is the same for both OpenSSH and WinSCP.
The file will be owned by idcuser after copying with SCP but the WebSphere user will be different. By default the WebSphere user is virtuser. On the WebSphere server machine change file owner and copy the file to the
monitored directory with the commands below (for Linux, use similar commands on Windows)
$ sudo mv migrationEAR.ear Profiles/AppSrv01/monitoredDeployableApps/servers/server1/.
My application is called migrationEAR.
simplest and most appropriate for the cloud.
The views in this blog are my own.