In the previous post Dustin shared a great video demonstrating the value of the IBM Image Construction and Composition Tool that is now delivered with IBM Workload Deployer V3.1. This is certainly one of the key new features of IBM Workload Deployer V3.1. However, there are also a number of other compelling enhancements and features that we would like communicate.
I created the attached video to highlight some of these features included in new Workload Deployer release. The video uses the web console to highlight some of the features and capabilities, giving a brief introduction for each one. Without going into a lot of depth, I think it gives a nice overview. This may be especially helpful if you already have Workload Deployer v3.0 and want to see the value you will get when you upgrade to Workload Deployer v3.1. Check it out.
We believe that these new features make IBM Workload Deployer V3.1 an even better solution for your private cloud needs. Please let us know what you think.
I recently read the Open Cloud Standards incubator charter proposed by the Distributed Management Task Force (DMTF), and I think this is a great effort to propel the cloud computing effort forward (read the charter here). In the interest of disclosure, I’m not just saying this because I happen to be employed by one of the supporters of the movement. It’s time to acknowledge that open standards are not an inhibitor to innovation, but instead they facilitate the kind of technological adoption that makes innovation both possible and profitable.
The incubator charter sets its eyes on standards around cloud resource management. Its main focus will concern the management of cloud computing elements that make up the Infrastructure as a Service (IaaS) layer, but it will also discuss some elements of the Platform as a Service (PaaS) layer. The charter lists among its deliverables a cloud taxonomy, cloud interoperability white paper, informational specifications, and security requirements for cloud resource management. The hope is that such standards would increase cloud interoperability, mitigate the potential for vendor lock-in, and guide companies who already deliver resource management products and are looking to extend such capabilities to cloud resource management.
In particular, there are a couple of deliverables that sound interesting. The first, a cloud taxonomy including terms and definitions, is a necessary step to move forward any cloud computing standardization efforts. There is too much disagreement among the cloud industry at large as to exactly what defines and makes up cloud computing. While healthy discussion and debate on this subject are good, it is time to agree on a standard definition that all can at least accept. Without a common, acceptable definition, standards movements will be crippled since it’s hard to govern something we cannot define.
The charter also proposes information specifications that would define profiles for the management of resources within a cloud computing environment. If these types of management standards were produced and subsequently adopted by a multitude of cloud providers, consumers could define a common cloud interaction layer and freely switch in and out the provider. In addition to these new specifications, updates to existing standards in DMTF are among the deliverables. The existing specification that is explicitly called out in the charter is the OVF packaging standard. I can only hope this means a standard packaging for virtual images deployed in a cloud environment. If that is indeed the goal, by combining a common cloud management mechanism with a common cloud packaging strategy, consumers would be provided the ultimate choice among cloud providers. They could package and deploy services meant to run in the cloud in such a way that the task would be repeatable across any cloud provider adopting the DMTF’s standards. In addition, the mention of updating existing specifications sends a clear message that cloud computing standardization efforts must not duplicate existing standards work. Instead, new specifications are introduced where necessary, and existing specifications are updated to accommodate this new computing paradigm.
I’m excited about the incubator charter announced by DMTF. I’m sure this is just the beginning in what will be a long journey of providing for an open cloud, but it is a necessary first step. I for one don’t buy the argument that open standards stifle innovation. Instead, I believe open standards increase technological adoption, and more consumers mean more opportunities, and demand, to distinguish offerings by creative innovation. What do you think about cloud computing standards? Let us know below, or send us an email.
I just wanted to point out a great opportunity for anybody considering leveraging IBM Workload Deployer v3 to deploy Database workloads. On June 29th Rav Ahuja, a Senior Product Manager for Data Management at IBM, will be hosting a webcast entitled "Easily Deploying Private Clouds for Database Workloads". He will be joined by Chris Gruber (Product Manager, Database as a Service), Leon Katsnelson (Program Director, IM Cloud Computing Center of Competence), and Sal Vella (Vice President, Database Development and Warehousing) in this panel discussion.
As many of you already know, IBM Workload Deployer v3 comes pre-loaded with DB2 images and patterns that are configured to rapidly provision standardized database servers for any number of purposes. The servers can be deployed in standalone configurations or as part of a complete virtual system including web components with the database components. These servers can also be configured for high availability scenarios. This panel discussion will cover all of these scenarios and more.
You can read more about the webcast in this blog post by Rav Ahuja.
If you want further details about how to build and rapidly deploy databases in a private cloud, be sure to attend this free webinar on June 29th.
In my opinion, declarative deployment models are key to the entire notion of Platform as a Service (PaaS). That is, users should concern themselves with what they want, but not necessarily how to get it. The PaaS system should be able to interpret imperatives from the user and automatically convert that to a running system. In this respect, I think the new virtual application pattern, and more specifically policies, in IBM Workload Deployer takes a giant leap toward a more declarative deployment model.
In IBM Workload Deployer, policies allow you to 'decorate' your virtual application pattern with functional and non-functional requirements. In other words, they provide a vehicle for you to tell the system what qualities of service you expect for your application environment. To put a little context around this discussion, let's examine the policies available in the virtual application pattern for web applications. Specifically, let's look at the four policy types you can attach to Enterprise Application, Web Application, and OSGI Application components in this pattern:
Scaling policy: When it comes to cloud, the first thing many folks think about is autonomic elasticity. Applications should scale up and down based on criteria defined by the user. Well, that is exactly what the scaling policy lets you do. You simply attach this policy to your application component, and then specify properties that define when to scale. First, you choose a scaling trigger from a list that includes application response time, CPU usage, JDBC connection wait time, and JDBC connection pool usage. After choosing your trigger, you decide the minimum and maximum number of application instances for your deployment, and then you choose the minimum number of seconds to wait for an add or remove action. At this point, you can deploy your application and IBM Workload Deployer will monitor the environment, automatically triggering scaling actions as needed.
JVM policy: I would be willing to bet that nearly all of you tune the JVM environment into which you deploy your applications. The JVM policy allows you to take two common tuning actions, setting the JVM heap sizes and passing in JVM arguments, as well as attach a debugger to the Java process (especially useful in development and test phases). You can also use the policy to enable verbose garbage collection (invaluable to understanding heap usage patterns for your application) and select the bit level (from 32 or 64) for your application. Again, all you have to do is attach the policy and specify the properties. IBM Workload Deployer will take care of the required configuration updates.
Routing policy: The routing policy provides a simple way to specify virtual hostnames and allowable protocols (HTTP or HTTPS) for your application. Attach the policy, provide the virtual hostname you want to use, select the desired protocols, and that's it! Remember, once you set the virtual hostname you will need to update your name server to map the hostname to the appropriate IP address.
Log policy: During the development and test phase, it is likely that you will want to enable certain trace strings in the application runtime. The log policy allows you to provide trace strings for your application, and it makes sure that the appropriate configuration updates occur in the deployed environment.
While this is not an exhaustive explanation of each of the policies above, I hope it gives you a basic idea of what they are and how to use them. To me, declarative deployment models are going to be a crucial part of making PaaS successful, so I am really excited about the notion of policies in IBM Workload Deployer. What do you think?
When one uses IBM Workload Deployer (previously WebSphere CloudBurst) to deploy a virtual system pattern, they benefit from a completely automated deployment process. The automation includes the creation and placement of virtual machines, injection of IP addresses, initiation of internal processes, and invocation of included scripts. Most of these processes are straightforward and require little more than a brief overview. However, the placement of virtual machines stands out, and it's inner workings are the subject of quite a few questions when I discuss the appliance. With that in mind, I thought I would provide a little more information on how the placement algorithm in IBM Workload Deployer works.
The placement subsystem in IBM Workload Deployer considers three primary elements: compute resource, availability, and license optimization. Compute resource availability is the gating factor for placement. That means that IBM Workload Deployer first looks at the available CPU, memory, and storage resource in the collection of hypervisors making up the cloud group(s) you are targeting for deployment. If a particular hypervisor cannot provide enough resource based on the amount you requested for your deployment, then it is automatically taken out of the eligible hosts pool. It is important to note that IBM Workload Deployer will overcommit CPU, and it will overcommit storage if you direct it to do so. It will not overcommit memory because that could severely degrade the performance of the application(s) running in the virtual machines.
After choosing the pool of hypervisors that are capable of hosting the virtual machines in your deployment from a compute resource perspective, the appliance then considers high availability. To better understand this particular placement stage, let's consider an example. Consider you are deploying a pattern based on WebSphere Application Server Hypervisor Edition and it contains two custom node parts. It is conceivable, and in fact likely, that these two custom node parts will host members of the same cluster, and thus the two nodes will support the same applications. As such, IBM Workload Deployer will attempt to place the two custom nodes on different physical machines in order to prevent a single point of failure. Of course, this depends on having two hypervisors with enough resource (CPU, memory, storage) to host the virtual machines, but the appliance makes that decision in the first placement stage.
After considering compute resource and high availability, IBM Workload Deployer moves to the last stage of placement: license optimization. In this stage, the placement subsystem attempts to place the virtual machines on hypervisors in a way that minimizes the licensing cost to you. The appliance can do this because it is aware of IBM virtualization licensing rules and takes those into account during this stage (if you aren't familiar with virtualization licensing rules and you are curious, ask you're sales representative to explain some time). During this stage, it will not violate any resource overcommit directives or rules in place, nor will it compromise system availability, but it will seek to minimize costs within these parameters.
At this point, I should make something clear that may already have occurred to you. You can override most of these placement rules by creating a cloud group containing only one hypervisor. In this case, IBM Workload Deployer will put all virtual machines on the single hypervisor until it runs out of compute resource (memory is likely to be the constraining factor). I would not suggest that you do this unless you have a good reason or you are in a simple pilot phase, but I do like to point out the art of the possible!
While not incredibly deep from a technical perspective, I do hope that this provided a few helpful details on what goes on during the placement stages of deployment. If you have any questions, do let me know.
This week is a busy week getting ready for IMPACT next week. I'm looking forward to the conference, and I thought I would share a few things on my agenda. Naturally, my agenda includes the sessions I am running:
10:15 AM - 11:30 AM
TDC-2973A Meet the Experts and Demo: WebSphere Cloudburst Appliance
Come and meet the experts responsible for the WebSphere Cloudburst Appliance, and see a demo of its functionality in this informal setting.
1:30 PM - 4:30 PM
TDC-1369A Lab: Working with the WebSphere CloudBurst Appliance
Come and work hands on with the WebSphere CloudBurst Appliance to create your own WebSphere application environments in a cloud. The lab will guide you through using WebSphere CloudBurst to create and deploy WebSphere virtual systems in a private cloud. Youll learn how to create custom WebSphere and DB2 topologies by extending virtual images, creating patterns, and using scripts. You'll get a chance to work with the easy to use Web 2.0 user interface. Youll be amazed at the ease of use WebSphere CloudBurst brings to configuring, deploying, and running WebSphere environments in a private cloud.
1:30 PM - 2:45 PM
TAD-1370A Simplifying Development using Rational Tools with WebSphere CloudBurst Appliance
Are you looking to really simplify your WebSphere development and test environments - including never needing to install or configure WebSphere again? If so, come hear about how you can use the IBM WebSphere CloudBurst Appliance along with Rational tools like the Rational Automation Framework for WebSphere and Rational Software Architect to create a dynamic development and test cloud. With the integration of WebSphere CloudBurst and selected Rational tools, you worry about the application development, while WebSphere CloudBurst worries about the WebSphere infrastructure and your cloud resources. Come to this combination of presentation and demo to see how easy development and testing can be.
In addition to these, there are some other exciting WebSphere CloudBurst sessions on tap:
3:45 PM - 5:00 PM
TDC-2498A WebSphere CloudBurst Appliance at Lowe's
Lowe's is evaluating WebSphere CloudBurst Appliance (WCA) as a tool for managing their X86 and PowerVM environments in a cloud fashion. Come to hear how Lowes believes WCA fits into an enterprise companys cloud strategy. This session will discuss the work done at Lowes so far and the use cases planned for WCA at Lowe's. Attendees can understand how WCA is delivering value in an adopter's environment.
5:15 PM - 6:30 PM
TDC-1368A Introduction to WebSphere CloudBurst Appliance
The WebSphere CloudBurst Appliance delivers capabilities to create, monitor, and maintain private WebSphere clouds. It provides you the capability to quickly and simply create, deploy, and maintain virtualized WebSphere application environments running on a heterogeneous, shared pool of resources that make up your cloud. In this session, we will provide an overview of the WebSphere CloudBurst Appliance features and benefits and demonstrate the latest capabilities.
1:30 PM - 2:45 PM
TDC-1758A Building Private Clouds with WebSphere CloudBurst Appliance
Come join us as we discuss how the WebSphere development and test organization built a large private cloud from the ground up using WebSphere CloudBurst Appliance. We have lowered the entry requirement to get a meaningful WebSphere Application Server development environment (days down to minutes), saved costs by improving hardware utilization while simplifying our management of physical resources and topologies. We will discuss best practices for adhering to security requirements, creating reusable automation scripts for your applications and configurations and maintaining your cloud. Allow us to share our experience in using WebSphere CloudBurst Appliance to create our automated regression infrastructure, and to provide up-to-date deployments to our test team.
4:45 PM - 6:00 PM
TDC-1946A BSkyB's Experiences using the WebSphere CloudBurst Appliance V1.1
At Impact 2009, IBM announced the launch of the WebSphere CloudBurst Appliance. BSkyB witnessed this launch and were very keen to understand the device's potential. This presentation details their experiences to date, and their vision for incorporating the appliance into their organization. Details will include bringing the device in house, setting up the cloud, and doing deployments. BSkyB will also discuss the customisation process, and how they used the extend / capture and scripting capabilities to add content including WebSphere Process Server. The presenters will share their lessons learned as they continue their journey using WebSphere CloudBurst for agile environment provisioning and simplified WebSphere Administration.
10:15 AM - 11:30 AM
TDC-2063A Panel: WebSphere CloudBurst Appliance Customers Describe their Experiences
A panel of several customers who have adopted WebSphere CloudBurst Appliance will discuss their experiences with the product, and answer questions related to their experiences.
9:00 AM - 10:15 AM
TDC-1884A Using WebSphere CloudBurst Appliance in a PowerVM Environment
This session will discuss the concepts and issues associated with implementing the WebSphere CloudBurst Appliance (WCA) in a PowerVM environment. The components of the implementation including VMControl, IBM System Director, HMC, NIM. and WebSphere CloudBurst will be explained, along with their relationships and functions. This in-depth session will also provide best practices from early adopter deployments and performance experiences.
1:30 PM - 2:45 PM
TBR-2491A Customizing a Private Cloud for WebSphere Process Server Applications
Every enterprise has a unique set of standards when it comes to the applications that are deployed and the qualities of service that are required for those applications. Come to this session to learn some of the best practices around pattern customization and maintenance of the images in the WebSphere CloudBurst Appliance for your specific requirements. We will use the creation of a WebSphere Process Server double gold topology pattern to show these best practices. This session will also cover the practices involved with maintaining these patterns.
As you can see there is going to be quite a bit of activity around WebSphere CloudBurst at IBM IMPACT 2010. The lists above is not all encompassing either. Visit the IBM Impact site for more information. If you are registered to attend, be sure to visit the agenda builder website for the conference.
The reason I suggest the application proxy approach is twofold. First, it affords you the ability of having custom interactions with the REST API. For instance, you may insert logic into the server-side proxy code that returns only a subset of the JSON data contained in the response from the appliance. Alternatively, in an effort to reduce the chattiness on your client-side, you may join JSON data from multiple different REST requests to the appliance to fulfill a single client request. You may even decide to represent the data in an all together different format than JSON. All of these options and many more are available to you if you implement an application-based proxy to the REST API.
The second reason I suggest the application approach is that it is easier, and seemingly safer, to not deal with user passwords on the client-side. If you setup your application proxy, you can configure it to retrieve the appropriate password from a secure location (like an encoded file) based on information passed along in the request. This means the password information is only present in the request (in encoded form of course) from the application proxy to the WebSphere CloudBurst Appliance.
The good news about the application-based proxy approach is that it is simple to put in place. I composed one using the open source Apache Wink project. The Apache Wink project is an open source implementation of the JAX-RS specification (and then some), and it enables you to develop POJOs that are in turn exposed in a RESTful manner. In my case, I had a single resource POJO:
The Apache Wink runtime routes any HTTP GET request whose path is like /resources/* to the getResources method in the WCAResource class. This method passes along information taken from the query string (the host name of the target WebSphere CloudBurst Appliance and the requesting WebSphere CloudBurst username), as well as the HTTP path information and sends it on to the getResource method declared as follows:
The getResource method above uses the WebSphere CloudBurst host name and the request path to construct the URL for the corresponding WebSphere CloudBurst REST API call. Next, it constructs an Apache Wink Resource object and sends the REST request along to the WebSphere CloudBurst Appliance. How do we authenticate this request? We use the WebSphere CloudBurst username (sent as a query string parameter) to retrieve the appropriate encoded password information. Once we have that, we construct the necessary header for basic authorization over SSL.
The application-based proxy shown here is simply a pass-through. It does not manipulate the data returned from the WebSphere CloudBurst REST API, nor does it map a single client-side call to multiple REST requests. However, it would be simple enough to extend it to do any of those things. If you have any questions about the code here, please let me know. I'd be happy to share more of the code, or talk about how and where to extend it.
One of the new features that debuted in WebSphere CloudBurst 1.1 is the ability to resize the disks in a virtual image during the extend and capture (image customization) process. If you remember, the virtual images that exist in the WebSphere CloudBurst catalog are made of multiple virtual disks. In WebSphere CloudBurst 1.0 a default size was used for the virtual disks and this could not be changed, even during the image extension process. To be quite honest we got quite a bit of feedback about this, and so with version 1.1 while default sizes are still provided, you can specify the eventual size of each of the virtual disks during the image extension process.
As an example, consider the WebSphere Application Server Hypervisor Edition virtual image. This image contains four virtual disks: one for the WebSphere Application Server binaries, one for the WebSphere Application Server profiles, one for the IBM HTTP Server, and one for the operating system. The default size of each of these disks in the 220.127.116.11 version of the image is 6GB, 2GB, 1GB, and 12GB respectively, for a total of roughly 21GB. While that may be fine for some, what happens if you are going to be installing various other third-party software packages in the image? You may need more disk space for the operating system's virtual disk. Perhaps your WebSphere applications produce log files of considerable size. In that case you may want to increase the default size of the WebSphere Application Server profiles disk space.
Those scenarios and more are exactly why the resizing capability was added. When you extend the WebSphere Application Server Hypervisor Edition 18.104.22.168 virtual image in WebSphere CloudBurst 1.1, you will be presented the option to resize one or more of the virtual disks:
In the case above the default operating system disk size is bumped up to 16GB from the default 12GB size. Also note that in addition to changing the disk size, you can specify the number of network interfaces for your custom image.
Obviously, when you increase the size of the disks within the virtual image you are also increasing the storage requirements for that image when it is deployed to a hypervisor. Keep this in mind when you are calculating the upper bound capacity of your cloud. If you want to see more about how this feature works, check out this video.
In my prior job at IBM, I was, on more than one occasion, reminded of the pains of dealing with software development tools. It seemed to be a constant battle to keep up with licenses, install critical fixes, and update to the latest version of whatever tool I happened to be using. Since I often worked on projects across multiple machines, I had to ensure that versions of the tool on different machines were reasonably close and that any code formatting settings were consistent across the different tool instances. On top of this, the tools were sometimes so CPU intensive that multitasking on the same machine running the tool was impossible.
All of the above pains were a direct function of the tools being installed on my local machine, so you can imagine my interest in a recent announcement by IBM signaling the launch of a pilot program offering Tools as a Service. The program, initially offered to students and faculty of selected universities, delivers hosted software development tools to developers. Users of the development tools do not install, maintain, or run the products on their local machine, instead they access them through a cloud maintained by IBM. The tools can be accessed from any machine with an internet connection, and a developer's sandbox is persisted across multiple sessions. The developer simply logs in, does work, and at some point saves his/her changes and logs out. The saved changes can be accessed at some point in the future from the same machine or an entirely different one.
This is exactly what I needed! Like many developers, I wanted to focus on writing code not maintaining a suite of tools. I for one hope this eventually extends beyond a pilot program and becomes a mainstream practice. You can read more about IBM's Tools as a Service initiative here.
If you've read anything I've written about WebSphere CloudBurst up to this point you know all about patterns. Using the appliance you can easily and quickly build, deploy, and manage these representations of your middleware application environments. Today, I want to focus in on the deployment piece in particular and take a look at how you can easily automate this process.
You can use the WebSphere CloudBurst web console to deploy patterns, and when doing so you can even schedule the deployment to happen at a later date. This scheduling capability certainly gets you on the road to an automated deployment process, but what if you want to take it one step further and eliminate the need for someone to login and manually move around the web console to schedule automated deployments? In this case, you can use either the CLI or the REST interface that WebSphere CloudBurst offers.
In this post I thought I'd take a look at using the CLI interface in order to set the stage for some nice automation around pattern deployment. It starts out with a properties file that provides details about my deployment. This includes the cloud to deploy to, the pattern to deploy, password information, and the time at which the virtual system should start.
SYSTEM_NAME_PREFIX=New App Development
TARGET_CLOUD=Default ESX group
TARGET_PATTERN=WebSphere single server
Imagine that the properties file above gets written as the result of some other action, such as the completion of your application's build process. With the properties file in place, and I'll point out that your properties file can and probably will be more robust than above, let's move on to the code that handles the deployment process based on the information in said file. First, we have a small amount of CLI code to retrieve and parse the input data (I omitted the straight-forward properties retrieval for space):
from datetime import datetime, timedelta
from java.util import Properties
from java.io import FileInputStream
// read in and retrieve properties using java.util.Properties API (i.e. props.getProperty('DEPLOYMENT_DATE'))
parsedParts = deploymentDate.split(" ")
systemName = systemName + "_" + deploymentDate
dateParts = parsedParts.split("/")
timeParts = parsedParts.split(":")
monthPart = int(dateParts)
dayPart = int(dateParts)
yearPart = int(dateParts)
hourPart = int(timeParts)
minutePart = int(timeParts)
Next is the code that actually schedules the pattern deployment:
First we get the desired deployment time and current time as datetime objects. After that, assuming the desired deployment time has not already elapsed, we calculate the difference between the desired deployment time and current time. This difference, in seconds, is then added to the result of the time.time() value to come up with a start time. After that is done, we simply retrieve the cloud that was indicated in the properties file, and then we call the runInCloud method for the pattern indicated. When calling the runInCloud method we supply the name of the virtual system that will be created, password information, and the start time we calculated earlier. As a result of this method call, a task will be generated in the target WebSphere CloudBurst Appliance and the virtual system will be started at the specified time. This will happen in an automated fashion with no human intervention required.
That's really all there is to automating the pattern deployment process using the CLI. In a more complete, end-to-end scenario you may envision the completion of one process, such as an application build process mentioned above, result in the writing of the properties file and in turn the call into the CLI to deploy a pattern. As always, feel free to send me any comments or questions.
In WebSphere CloudBurst, a script package is your vehicle to provide custom middleware configuration. This may mean installing applications, configuring application dependencies, or otherwise tuning the middleware layer. Script packages are essentially ZIP files that include some executable (shell script, wsadmin script, Java program, etc.), and optionally, artifacts that support the execution of the script. As was the intention, you can achieve just about anything you want with a script package. This allows you to be as flexible and creative as you need to be, but it can also leave you asking "Where do I start?" In this post, I want to take an in-depth look at constructing and using a script package in WebSphere CloudBurst.
Specifically, I want to create a script package that supplies configuration functionality for something I believe a fair number of you do: change the default ports used in WebSphere Application Server. To create this and deploy a pattern using the script package, I do the following:
Create a shell script that configures the desired ports
Add the new script as a WebSphere CloudBurst script package
Create a pattern with the new script package
Deploy the pattern and verify the result
First things first. I create the following shell script that configures the ports:
The script uses documented ANT commands included with the WebSphere Application Server to update the ports based on a starting port number. You will notice the script first sources the /etc/virtualimage.properties file. This file is automatically created by WebSphere CloudBurst on every virtual machine it starts. The file is a key/value file with basic information about the WebSphere cell such as the install root ($WAS_INSTALL_ROOT), the profile name ($PROFILE_NAME), host name ($HOSTNAME), and more. For a full list of the data that WebSphere CloudBurst includes in this file, check out this documentation.
In addition to utilizing the standard set of variables provided by WebSphere CloudBurst, my script above also makes use of the $STARTING_PORT variable. Obviously this variable is not in the standard set. In fact, I define the STARTING_POINT variable when I define my new script package in WebSphere CloudBurst.
First I zip up the shell script above and attach it to the new script package. Next, I tell WebSphere CloudBurst where to unzip the script package on the virtual machine, how to invoke the included script, and the name of any parameters to associate with the script. Once that is done I can use the script package in a new pattern.
For the sake of simplicity here, I create a new pattern by cloning an existing WebSphere Application Server single server pattern. I drag and drop the new Configure ports script package on the single part and end up with the pattern shown below.
Now I am ready to deploy the pattern by clicking the Deploy button. During the deployment process I configure each part in the pattern (in this there is only a single part). I supply configuration information like virtual memory allocation, WebSphere cell name, WebSphere node name, and password information. In addition, I also supply a value for the STARTING_PORT parameter that is part of the Configure ports script package included in the pattern. The value I supply here will get inserted into the /etc/virtualimage.properties file on the virtual machine, and the value's key will be STARTING_PORT.
Once the configuration information is supplied, I click OK on the configuration panel and deployment panel, and WebSphere CloudBurst goes about standing up my virtualized WebSphere cell and running my script to configure the ports for the server instance. When it is done, I login to the WebSphere Application Server administration console to verify my results. To do this, I navigate to the configuration for the single application server instance, and pull up its port definitions.
Based on the results I can see my customizations took effect. I successfully captured my own unique WebSphere environment (in this case with a custom port range) in the form of a pattern. This custom environment can be deployed as many times as I need, in an automated fashion, and I'm guaranteed consistent results each and every time.
I hope this gives you a better idea of what script packages are all about and how they can utilize both WebSphere CloudBurst and user-supplied data that exists in the /etc/virtualimage.properties file of each virtual machine. If you have any questions let me know. I'm on Twitter @damrhein, or you can leave a comment right here.
When it comes to building and using WebSphere CloudBurst patterns, people always ask me if I have any best practices. It turns out, I do. In fact, I have a singular piece of advice that wraps it all up: Build WebSphere CloudBurst patterns in a way such that once deployed, there is no after-the-fact, manual configuration for the running environment. That means, build the pattern so that it not only contains all the nodes necessary for your application environment, but it also contains all the configuration necessary for the environment.
Put like this, most everyone I talk to agrees with me. However, they quickly recognize that, absent this really cool integration with Rational Automation Framework for WebSphere, this means they will be writing scripts for many configuration actions and including them in patterns in the form of script packages. For users not familiar with configuration scripting for our WebSphere products, this can be a daunting proposition. But... it shouldn't be!
Recently, I put together a short presentation that lays out an iterative approach for developing script packages for WebSphere CloudBurst. Specifically, the presentation focuses on developing configuration script packages for the WebSphere Application Server (though the general concepts apply to all Hypervisor Edition products equally). I believe this method is useful for anyone, from novice users to WebSphere scripting gurus. The basic process goes something like this:
Identify: Identify the target WebSphere Application Server topology and configuration for your application environment.
Deploy: Build a WebSphere CloudBurst pattern that matches your desired topology and deploy it to your cloud.
Develop and Test: Develop and test your configuration script. Not a WebSphere Application Server scripting ninja? No worries. Use the Command Assistance feature in the WebSphere Application Server v7 administration console. This feature shows you the wsadmin commands that match the actions you manually take in the console. This affords a lower barrier of entry for those not familiar with wsadmin.
Package: Package up the resulting scripts into a script package along with metadata that describes the package.
Modify and redeploy: Load the new script package into your appliance, add it to your pattern, and then redeploy. Upon deployment completion, verify the scripts produce the desired result.
The presentation provides detail on the above steps and walks through an example scenario for this process. I am embedding it below, and I hope it proves useful. As always, feel free to send in any questions or comments.
I point this out about script packages because recently I put one together that is not a WebSphere Application Server administration task, but does provide configuration logic common to many WAS deployments. Specifically, I put together a script package that configures an IBM HTTP Server to be a reverse proxy server. This of course, allows clients to send requests to the IBM HTTP Server and have those requests pass through to a specified back-end destination (i.e. a service hosted on WebSphere Application Server) based on URI paths.
The script package is not all that different from many of the ones I put together. It contains a shell script (which provides configuration and orchestration logic) as well as a cbscript.json file that defines the script package's characteristics when I upload it into WebSphere CloudBurst. The notable difference in this script package is that I include a Perl script that modifies the IBM HTTP Server configuration file. This just reiterates the point that you are not limited to only wsadmin and shell scripts within your script packages.
The workings of the script package are quite straightforward. It starts with a call to the shell script that modifies the IBM HTTP Server's configuration file to ensure the loading of a couple proxy modules:
sed -i s/"#LoadModule proxy_module modules\/mod_proxy.so"/"LoadModule proxy_module modules\/mod_proxy.so"/g $HTTP_CONF
sed -i s/"#LoadModule proxy_http_module modules\/mod_proxy_http.so"/"LoadModule proxy_http_module modules\/mod_proxy_http.so"/g $HTTP_CONF
As you can see, the createProxy.pl script accepts a single argument. This argument represents the reverse proxy configuration information provided by the user during deployment. The Perl script parses the single argument and creates the appropriate proxy directives in the IBM HTTP Server's configuration file:
After the invocation of the script above, control returns to the shell script. The shell script restarts the IBM HTTP Server so that the configuration changes take place. The result is an up and running IBM HTTP Server acting as a reverse proxy based on information supplied during deployment. The listings here do not show the full script package, but I hope to have it up on our WebSphere CloudBurst Samples Gallery soon.
If the script above provides some configuration logic you can use, that is good. However, my main point for bringing it up here is to point out that WebSphere CloudBurst script packages can be more than shell and wsadmin scripts that perform WebSphere Application Server configuration tasks. You can use them to do any sort of scripted activity that is essential to your application middleware deployment process. Happy scripting!
Many enterprise IT decision makers are probably currently, or will be shortly, contemplating some type of cloud computing solution that promises to deliver benefits to their operation. After all, cloud computing solutions, when utilized prudently, are hard to resist. As these decision makers prepare to weigh the pros and cons of such solutions, there is one consideration that is perhaps more important than all others: Will this solution enable elasticity among my services and applications?
Hopefully the answer to the above question is a confident 'Yes'. A lot of discussion is given to providing virtualized, dynamically scaled resources such as servers, processors, operating systems, etc. and rightfully so. By optimizing the usage and consumption of these infrastructure resources, IT operations can be significantly enhanced. However, these resources are often times just a mean to an end, that end being service delivery. IT provides services and applications to end-users that are critical in terms of revenue generation or optimal business activity. These are the resources that most need elastic capabilites to ensure that end-users are continually provided responsive service. Cloud computing solutions need to address this capability by providing the means to package services in manageable units and deliver the ability to govern the scale of these units in an autonomic fashion.
Note, that in the question above I used the word enable. This was deliberate since it is hard to comprehend how any cloud computing solution could provide "out-of-the-box" service elasticity. The implementation of a SOA architecture becomes a necessary partner to achieving true service elasticity. Services must be developed in such a way that they are loosely coupled from other services and required components such as databases. Otherwise, the ability of one service to dynamically scale may be implicitly connected to another service's ability to achieve such elasticity. In this type of intertwined architecture, maximum elasticity will never be achieved.
What do you think about the current cloud landscape? Is there enough focus on service elasticity, and is service elasticity even the most valuable aspect of cloud computing? Leave a comment below, or reply to us on Twitter @WebSphereClouds.
If you are going to install and use WebSphere CloudBurst in your own environment, it is very likely that you would want at least two appliances. Perhaps you want to have a standby appliance in case of a failure on the main appliance, or maybe you have different teams that are looking to utilize the appliance in different data centers. In any case, once you install multiple appliances there's another requirement that will pop up pretty quickly. Naturally you are going to want to share custom artifacts among the various WebSphere CloudBurst boxes.
When I say custom artifacts, namely I mean virtual images, patterns, and script packages. Script packages have been easy enough to share since WebSphere CloudBurst 1.0 because you can simply download the ZIP file from one appliance and upload it to another. However, there are some enhancements in WebSphere CloudBurst 1.1 that make it easy to share both patterns and images among your different appliances.
As far as patterns go, there is a new script included in the samples directory of the WebSphere CloudBurst command line interface package called patternToPython.py. This script will transform a pattern you specify into a python script. The resulting python script can then be run against a different WebSphere CloudBurst (using the CLI), and the result is the pattern is created on the target appliance. You need to be sure that the artifacts that pattern references (script packages and virtual images) exist on the target appliance and have the exact same name as they do on the appliance from which the pattern was taken. There are no other caveats, and this new sample script makes it really simple to move patterns between appliances.
For virtual images, a new feature was added that allows you to export a virtual image from the WebSphere CloudBurst console. Simply select a virtual image, specify a remote machine (any machine with SCP enabled), and click a button to export the image as an OVA file. This OVA file can then be added to another WebSphere CloudBurst catalog using the normal process for adding virtual images. You can see this feature in action here.
Stay tuned for more information about some of the handy new features in WebSphere CloudBurst 1.1. We also should have a comprehensive look at the new release coming soon in a developerWorks article.