In the previous post Dustin shared a great video demonstrating the value of the IBM Image Construction and Composition Tool that is now delivered with IBM Workload Deployer V3.1. This is certainly one of the key new features of IBM Workload Deployer V3.1. However, there are also a number of other compelling enhancements and features that we would like communicate.
I created the attached video to highlight some of these features included in new Workload Deployer release. The video uses the web console to highlight some of the features and capabilities, giving a brief introduction for each one. Without going into a lot of depth, I think it gives a nice overview. This may be especially helpful if you already have Workload Deployer v3.0 and want to see the value you will get when you upgrade to Workload Deployer v3.1. Check it out.
We believe that these new features make IBM Workload Deployer V3.1 an even better solution for your private cloud needs. Please let us know what you think.
If you've seen the official demo on ibm.com/cloudburst then one of these demos will be familiar to you. However, there is a three part deep dive series that is being made public for the first time. The deep dive walks users through the different features that WebSphere CloudBurst offers to create, deploy, and manage WebSphere virtual systems in their private cloud.
I think the demos give a good idea as to how WebSphere CloudBurst provides the features we've been talking about on this blog. If you have questions after watching the demos visit our forum and start or participate in a thread.
I recently read the Open Cloud Standards incubator charter proposed by the Distributed Management Task Force (DMTF), and I think this is a great effort to propel the cloud computing effort forward (read the charter here). In the interest of disclosure, I’m not just saying this because I happen to be employed by one of the supporters of the movement. It’s time to acknowledge that open standards are not an inhibitor to innovation, but instead they facilitate the kind of technological adoption that makes innovation both possible and profitable.
The incubator charter sets its eyes on standards around cloud resource management. Its main focus will concern the management of cloud computing elements that make up the Infrastructure as a Service (IaaS) layer, but it will also discuss some elements of the Platform as a Service (PaaS) layer. The charter lists among its deliverables a cloud taxonomy, cloud interoperability white paper, informational specifications, and security requirements for cloud resource management. The hope is that such standards would increase cloud interoperability, mitigate the potential for vendor lock-in, and guide companies who already deliver resource management products and are looking to extend such capabilities to cloud resource management.
In particular, there are a couple of deliverables that sound interesting. The first, a cloud taxonomy including terms and definitions, is a necessary step to move forward any cloud computing standardization efforts. There is too much disagreement among the cloud industry at large as to exactly what defines and makes up cloud computing. While healthy discussion and debate on this subject are good, it is time to agree on a standard definition that all can at least accept. Without a common, acceptable definition, standards movements will be crippled since it’s hard to govern something we cannot define.
The charter also proposes information specifications that would define profiles for the management of resources within a cloud computing environment. If these types of management standards were produced and subsequently adopted by a multitude of cloud providers, consumers could define a common cloud interaction layer and freely switch in and out the provider. In addition to these new specifications, updates to existing standards in DMTF are among the deliverables. The existing specification that is explicitly called out in the charter is the OVF packaging standard. I can only hope this means a standard packaging for virtual images deployed in a cloud environment. If that is indeed the goal, by combining a common cloud management mechanism with a common cloud packaging strategy, consumers would be provided the ultimate choice among cloud providers. They could package and deploy services meant to run in the cloud in such a way that the task would be repeatable across any cloud provider adopting the DMTF’s standards. In addition, the mention of updating existing specifications sends a clear message that cloud computing standardization efforts must not duplicate existing standards work. Instead, new specifications are introduced where necessary, and existing specifications are updated to accommodate this new computing paradigm.
I’m excited about the incubator charter announced by DMTF. I’m sure this is just the beginning in what will be a long journey of providing for an open cloud, but it is a necessary first step. I for one don’t buy the argument that open standards stifle innovation. Instead, I believe open standards increase technological adoption, and more consumers mean more opportunities, and demand, to distinguish offerings by creative innovation. What do you think about cloud computing standards? Let us know below, or send us an email.
I just wanted to point out a great opportunity for anybody considering leveraging IBM Workload Deployer v3 to deploy Database workloads. On June 29th Rav Ahuja, a Senior Product Manager for Data Management at IBM, will be hosting a webcast entitled "Easily Deploying Private Clouds for Database Workloads". He will be joined by Chris Gruber (Product Manager, Database as a Service), Leon Katsnelson (Program Director, IM Cloud Computing Center of Competence), and Sal Vella (Vice President, Database Development and Warehousing) in this panel discussion.
As many of you already know, IBM Workload Deployer v3 comes pre-loaded with DB2 images and patterns that are configured to rapidly provision standardized database servers for any number of purposes. The servers can be deployed in standalone configurations or as part of a complete virtual system including web components with the database components. These servers can also be configured for high availability scenarios. This panel discussion will cover all of these scenarios and more.
You can read more about the webcast in this blog post by Rav Ahuja.
If you want further details about how to build and rapidly deploy databases in a private cloud, be sure to attend this free webinar on June 29th.
In my opinion, declarative deployment models are key to the entire notion of Platform as a Service (PaaS). That is, users should concern themselves with what they want, but not necessarily how to get it. The PaaS system should be able to interpret imperatives from the user and automatically convert that to a running system. In this respect, I think the new virtual application pattern, and more specifically policies, in IBM Workload Deployer takes a giant leap toward a more declarative deployment model.
In IBM Workload Deployer, policies allow you to 'decorate' your virtual application pattern with functional and non-functional requirements. In other words, they provide a vehicle for you to tell the system what qualities of service you expect for your application environment. To put a little context around this discussion, let's examine the policies available in the virtual application pattern for web applications. Specifically, let's look at the four policy types you can attach to Enterprise Application, Web Application, and OSGI Application components in this pattern:
Scaling policy: When it comes to cloud, the first thing many folks think about is autonomic elasticity. Applications should scale up and down based on criteria defined by the user. Well, that is exactly what the scaling policy lets you do. You simply attach this policy to your application component, and then specify properties that define when to scale. First, you choose a scaling trigger from a list that includes application response time, CPU usage, JDBC connection wait time, and JDBC connection pool usage. After choosing your trigger, you decide the minimum and maximum number of application instances for your deployment, and then you choose the minimum number of seconds to wait for an add or remove action. At this point, you can deploy your application and IBM Workload Deployer will monitor the environment, automatically triggering scaling actions as needed.
JVM policy: I would be willing to bet that nearly all of you tune the JVM environment into which you deploy your applications. The JVM policy allows you to take two common tuning actions, setting the JVM heap sizes and passing in JVM arguments, as well as attach a debugger to the Java process (especially useful in development and test phases). You can also use the policy to enable verbose garbage collection (invaluable to understanding heap usage patterns for your application) and select the bit level (from 32 or 64) for your application. Again, all you have to do is attach the policy and specify the properties. IBM Workload Deployer will take care of the required configuration updates.
Routing policy: The routing policy provides a simple way to specify virtual hostnames and allowable protocols (HTTP or HTTPS) for your application. Attach the policy, provide the virtual hostname you want to use, select the desired protocols, and that's it! Remember, once you set the virtual hostname you will need to update your name server to map the hostname to the appropriate IP address.
Log policy: During the development and test phase, it is likely that you will want to enable certain trace strings in the application runtime. The log policy allows you to provide trace strings for your application, and it makes sure that the appropriate configuration updates occur in the deployed environment.
While this is not an exhaustive explanation of each of the policies above, I hope it gives you a basic idea of what they are and how to use them. To me, declarative deployment models are going to be a crucial part of making PaaS successful, so I am really excited about the notion of policies in IBM Workload Deployer. What do you think?
Over time, many of our users learn to effectively leverage WebSphere CloudBurst user roles and fine-grained access controls to map activities and responsibilities in the appliance to the appropriate people and teams within their organization. Using these controls, they are able to define actions that a user or group can take, and they can define the set of resources on which they can take those actions. It is efficient, flexible, and an absolute necessity in many enterprise scenarios.
In some cases though, I talk with users that want a little more control, or probably better put, governance over the actions a user can take within a given role. Most often, this need arises when the discussion of pattern authoring comes up. If you want a user in WebSphere CloudBurst to be able to create patterns, you simply give them the Create new patterns permission. Once you give them that permission, the user can create patterns using both virtual image parts and script packages in the catalog. For many of the users I talk with, this approach suits their needs.
However, in some scenarios administrators want a little more insight and control over how the pattern authors build their patterns. Specifically, they want to ensure that patterns contain only approved virtual image parts and script packages. While you can certainly use the fine-grained access controls of the appliance to expose only the 'approved' virtual image parts and script packages, that alone may not be enough. After all, the definition of what is 'approved' may be different when building a pattern for testing purposes versus one built for production purposes. If the same pattern author builds both of those patterns, fine-grained access controls do not help as much. So, what can you do?
Have I ever told you how much I love the WebSphere CloudBurst CLI? It's powerful, easy to use, and a great automation enabler. It is also the perfect tool for our problem above. If you are looking to enforce certain constraints on WebSphere CloudBurst patterns, I strongly recommend using the CLI as a governance tool.
To provide a concrete example of what I mean, let's take a look at a generic pattern checking script I am working on now (I hope to have this in the samples gallery soon). Consider the case that I want to check that all of my test patterns for a specific application environment contain 1 deployment manager and between 1 and 3 custom nodes. In addition, I want to make sure that the parts for these nodes come from an approved virtual image, and I want to verify that the deployment manager contains the correct application installation script package. With the script I am currently writing, you would start by encapsulating this information in a properties file.
PatternAssertion_1=Customer Processing Test Environments
PatternAssertion_1_Requirements=Deployment manager:1:415:Install customer process app;Custom nodes:1-3:415
In the above, the PatternAssertion_1 key provides a name for the pattern verification assertion. The PatternAssertion_1_Requirements key provides the requirements for the pattern. The above requirements indicate that for a pattern to meet the assertion, it must contain 1 deployment manager part from the virtual image with reference number 415. In addition, the deployment manager must contain a script named Install customer process app. A valid pattern must also contain 1 to 3 custom node parts, also based on the virtual image with reference number 415. When done defining my requirements in the properties file, I simply invoke a script and pass in the file. As a result, I get information about which patterns satisfy or do not satisfy the assertions. For example:
The Customer Process Application pattern satisfied the requirements of the Customer Processing Test Environments assertion.
OR The Customer Process Application pattern did not satisfy the requirements of the Customer Processing Test Environments assertion
due to the following reason: The pattern is required to have a minimum of 1 and a maximum of 3 Custom node part(s), but it had 4.
As I said, I hope to have this sample script posted to the samples gallery soon. I am going through some final revisions and enhancements that I hope make it better and more generally applicable. In the meantime, I wanted to point out that pattern governance is indeed doable, and in fact not very hard to achieve with the CLI. I will be sure to post an update when the sample script is ready. In the meantime, let me know if you have any questions or comments.
When one uses IBM Workload Deployer (previously WebSphere CloudBurst) to deploy a virtual system pattern, they benefit from a completely automated deployment process. The automation includes the creation and placement of virtual machines, injection of IP addresses, initiation of internal processes, and invocation of included scripts. Most of these processes are straightforward and require little more than a brief overview. However, the placement of virtual machines stands out, and it's inner workings are the subject of quite a few questions when I discuss the appliance. With that in mind, I thought I would provide a little more information on how the placement algorithm in IBM Workload Deployer works.
The placement subsystem in IBM Workload Deployer considers three primary elements: compute resource, availability, and license optimization. Compute resource availability is the gating factor for placement. That means that IBM Workload Deployer first looks at the available CPU, memory, and storage resource in the collection of hypervisors making up the cloud group(s) you are targeting for deployment. If a particular hypervisor cannot provide enough resource based on the amount you requested for your deployment, then it is automatically taken out of the eligible hosts pool. It is important to note that IBM Workload Deployer will overcommit CPU, and it will overcommit storage if you direct it to do so. It will not overcommit memory because that could severely degrade the performance of the application(s) running in the virtual machines.
After choosing the pool of hypervisors that are capable of hosting the virtual machines in your deployment from a compute resource perspective, the appliance then considers high availability. To better understand this particular placement stage, let's consider an example. Consider you are deploying a pattern based on WebSphere Application Server Hypervisor Edition and it contains two custom node parts. It is conceivable, and in fact likely, that these two custom node parts will host members of the same cluster, and thus the two nodes will support the same applications. As such, IBM Workload Deployer will attempt to place the two custom nodes on different physical machines in order to prevent a single point of failure. Of course, this depends on having two hypervisors with enough resource (CPU, memory, storage) to host the virtual machines, but the appliance makes that decision in the first placement stage.
After considering compute resource and high availability, IBM Workload Deployer moves to the last stage of placement: license optimization. In this stage, the placement subsystem attempts to place the virtual machines on hypervisors in a way that minimizes the licensing cost to you. The appliance can do this because it is aware of IBM virtualization licensing rules and takes those into account during this stage (if you aren't familiar with virtualization licensing rules and you are curious, ask you're sales representative to explain some time). During this stage, it will not violate any resource overcommit directives or rules in place, nor will it compromise system availability, but it will seek to minimize costs within these parameters.
At this point, I should make something clear that may already have occurred to you. You can override most of these placement rules by creating a cloud group containing only one hypervisor. In this case, IBM Workload Deployer will put all virtual machines on the single hypervisor until it runs out of compute resource (memory is likely to be the constraining factor). I would not suggest that you do this unless you have a good reason or you are in a simple pilot phase, but I do like to point out the art of the possible!
While not incredibly deep from a technical perspective, I do hope that this provided a few helpful details on what goes on during the placement stages of deployment. If you have any questions, do let me know.
As many of you well know, virtual images are the foundation of virtual system patterns in IBM Workload Deployer. Whether you are using IBM Hypervisor Edition images or custom-built images produced by the IBM Image Construction and Composition Tool, every virtual system pattern has at least one virtual image as part of its foundation. So, if virtual images are the foundation of virtual system patterns, what is the foundation of these virtual images?
While you could probably make a good argument for a number of different things being the foundation of the virtual image (operating system, other installed software, etc.), I like to think that, at least in the context of IBM Workload Deployer, the activation engine inside the virtual image is the true foundation. Inside this activation engine, you will find a collection of scripts and services that are capable of configuring the virtual machine for use. Not only does this engine perform basic system-level actions like configuring the machine's hostname, IP address, time, and network interfaces, but it also configures the software on the inside of the virtual machine. For instance, the activation engine in the WebSphere Application Server Hypervisor Edition image is capable of fixing up profile information, federating nodes, creating application server clusters, and more. Best of all, in the case of IBM Hypervisor Edition images, you (the user) get all of this right out of the box. There is no logic to perform or administrative tasks to undertake in order for you to benefit from the activation engine. It is simply there!
So, at this point you may ask yourself 'If all of this is included right out of the box, why do I need to care?' That is a fair question, but ultimately I feel it is always important to understand the foundational elements of any technology. In this respect, I do not feel like the activation engine in the IBM Hypervisor Edition images is any different. Lately, I have been telling my users to take at least a little time to understand what the activation engine is and even more importantly, what it is doing for you during deployment. Specifically, I always suggest taking a little time to look at the scripts in the activation engine -- most often found in the /opt/IBM/AE/AS directory of a virtual machine deployed by IBM Workload Deployer.
What can be gained by taking the time to peruse through these scripts? I think most importantly, you will learn what the engine does for you and what you cannot do if you expect the image to deploy correctly. For instance, if you look in some of those activation engine scripts, you will see that it uses the sudo command in several places. While I know many of you may be tempted to remove the sudo command during extend and capture, if you do so it will break the activation engine. I have seen this happen multiple times, and trust me, if you did not know the activation engine used that command it is not necessarily an easy problem to debug. This is a case where the value of at least superficially understanding the activation engine is clear.
Want another example? Okay, consider that you want to run WebSphere Application Server as a user called wasadmin. At pattern deployment time, it is easy enough to supply wasadmin in the appropriate field of the part configuration data and click OK. IBM Workload Deployer deploys the system and voila, WebSphere Application Server is magically running as wasadmin. Everything is fine so far, but let's take this a step further and say that you previously performed an extend and capture, and you installed software components in the image that should be owned by your wasadmin user. It is technically possible to define users during extend and capture and then install software content via that user, but if you also want to specify that user as the WebSphere Application Server administrative user at deployment time, you will run into an issue. This is because the activation engine runs the usermod command during deployment to change the existing and default virtuser into the user that you specify -- in this case wasadmin. If the usermod command attempts to change virtuser to wasadmin but wasadmin already exists as a user on the operating system, the command will not complete properly, and it is very likely you will see further errors downstream. A simpler way to do this is to create the user during extend and capture, install any components via that user, and then delete the user before capturing. You can attach a deploy-time script that fixes up the appropriate settings for wasadmin (like user ID and group ID), and it will run after the activation engine successfully does a usermod and changes virtuser to wasadmin.Problem averted!
In reading some of the above, I fully realize that it may be a little confusing at first. That said, I assure you that there is not much to it at all once you have a basic understanding of the activation engine. With a basic understanding of the activation engine in tow, you will know what you do not need to do (e.g. create profiles, federate nodes, etc.), what you cannot do (e.g. remove the sudo command), and what you can do with a little bit of reconciliation work (e.g. define your WebSphere Application Server administrative user during image extension). I encourage you to take a little time with your next deployment and give the activation engine a once over. You will undoubtedly have a better understanding of the deployment process, and you will ultimately be in a position to most effectively leverage virtual system patterns in IBM Workload Deployer.
This week is a busy week getting ready for IMPACT next week. I'm looking forward to the conference, and I thought I would share a few things on my agenda. Naturally, my agenda includes the sessions I am running:
10:15 AM - 11:30 AM
TDC-2973A Meet the Experts and Demo: WebSphere Cloudburst Appliance
Come and meet the experts responsible for the WebSphere Cloudburst Appliance, and see a demo of its functionality in this informal setting.
1:30 PM - 4:30 PM
TDC-1369A Lab: Working with the WebSphere CloudBurst Appliance
Come and work hands on with the WebSphere CloudBurst Appliance to create your own WebSphere application environments in a cloud. The lab will guide you through using WebSphere CloudBurst to create and deploy WebSphere virtual systems in a private cloud. Youll learn how to create custom WebSphere and DB2 topologies by extending virtual images, creating patterns, and using scripts. You'll get a chance to work with the easy to use Web 2.0 user interface. Youll be amazed at the ease of use WebSphere CloudBurst brings to configuring, deploying, and running WebSphere environments in a private cloud.
1:30 PM - 2:45 PM
TAD-1370A Simplifying Development using Rational Tools with WebSphere CloudBurst Appliance
Are you looking to really simplify your WebSphere development and test environments - including never needing to install or configure WebSphere again? If so, come hear about how you can use the IBM WebSphere CloudBurst Appliance along with Rational tools like the Rational Automation Framework for WebSphere and Rational Software Architect to create a dynamic development and test cloud. With the integration of WebSphere CloudBurst and selected Rational tools, you worry about the application development, while WebSphere CloudBurst worries about the WebSphere infrastructure and your cloud resources. Come to this combination of presentation and demo to see how easy development and testing can be.
In addition to these, there are some other exciting WebSphere CloudBurst sessions on tap:
3:45 PM - 5:00 PM
TDC-2498A WebSphere CloudBurst Appliance at Lowe's
Lowe's is evaluating WebSphere CloudBurst Appliance (WCA) as a tool for managing their X86 and PowerVM environments in a cloud fashion. Come to hear how Lowes believes WCA fits into an enterprise companys cloud strategy. This session will discuss the work done at Lowes so far and the use cases planned for WCA at Lowe's. Attendees can understand how WCA is delivering value in an adopter's environment.
5:15 PM - 6:30 PM
TDC-1368A Introduction to WebSphere CloudBurst Appliance
The WebSphere CloudBurst Appliance delivers capabilities to create, monitor, and maintain private WebSphere clouds. It provides you the capability to quickly and simply create, deploy, and maintain virtualized WebSphere application environments running on a heterogeneous, shared pool of resources that make up your cloud. In this session, we will provide an overview of the WebSphere CloudBurst Appliance features and benefits and demonstrate the latest capabilities.
1:30 PM - 2:45 PM
TDC-1758A Building Private Clouds with WebSphere CloudBurst Appliance
Come join us as we discuss how the WebSphere development and test organization built a large private cloud from the ground up using WebSphere CloudBurst Appliance. We have lowered the entry requirement to get a meaningful WebSphere Application Server development environment (days down to minutes), saved costs by improving hardware utilization while simplifying our management of physical resources and topologies. We will discuss best practices for adhering to security requirements, creating reusable automation scripts for your applications and configurations and maintaining your cloud. Allow us to share our experience in using WebSphere CloudBurst Appliance to create our automated regression infrastructure, and to provide up-to-date deployments to our test team.
4:45 PM - 6:00 PM
TDC-1946A BSkyB's Experiences using the WebSphere CloudBurst Appliance V1.1
At Impact 2009, IBM announced the launch of the WebSphere CloudBurst Appliance. BSkyB witnessed this launch and were very keen to understand the device's potential. This presentation details their experiences to date, and their vision for incorporating the appliance into their organization. Details will include bringing the device in house, setting up the cloud, and doing deployments. BSkyB will also discuss the customisation process, and how they used the extend / capture and scripting capabilities to add content including WebSphere Process Server. The presenters will share their lessons learned as they continue their journey using WebSphere CloudBurst for agile environment provisioning and simplified WebSphere Administration.
10:15 AM - 11:30 AM
TDC-2063A Panel: WebSphere CloudBurst Appliance Customers Describe their Experiences
A panel of several customers who have adopted WebSphere CloudBurst Appliance will discuss their experiences with the product, and answer questions related to their experiences.
9:00 AM - 10:15 AM
TDC-1884A Using WebSphere CloudBurst Appliance in a PowerVM Environment
This session will discuss the concepts and issues associated with implementing the WebSphere CloudBurst Appliance (WCA) in a PowerVM environment. The components of the implementation including VMControl, IBM System Director, HMC, NIM. and WebSphere CloudBurst will be explained, along with their relationships and functions. This in-depth session will also provide best practices from early adopter deployments and performance experiences.
1:30 PM - 2:45 PM
TBR-2491A Customizing a Private Cloud for WebSphere Process Server Applications
Every enterprise has a unique set of standards when it comes to the applications that are deployed and the qualities of service that are required for those applications. Come to this session to learn some of the best practices around pattern customization and maintenance of the images in the WebSphere CloudBurst Appliance for your specific requirements. We will use the creation of a WebSphere Process Server double gold topology pattern to show these best practices. This session will also cover the practices involved with maintaining these patterns.
As you can see there is going to be quite a bit of activity around WebSphere CloudBurst at IBM IMPACT 2010. The lists above is not all encompassing either. Visit the IBM Impact site for more information. If you are registered to attend, be sure to visit the agenda builder website for the conference.
The reason I suggest the application proxy approach is twofold. First, it affords you the ability of having custom interactions with the REST API. For instance, you may insert logic into the server-side proxy code that returns only a subset of the JSON data contained in the response from the appliance. Alternatively, in an effort to reduce the chattiness on your client-side, you may join JSON data from multiple different REST requests to the appliance to fulfill a single client request. You may even decide to represent the data in an all together different format than JSON. All of these options and many more are available to you if you implement an application-based proxy to the REST API.
The second reason I suggest the application approach is that it is easier, and seemingly safer, to not deal with user passwords on the client-side. If you setup your application proxy, you can configure it to retrieve the appropriate password from a secure location (like an encoded file) based on information passed along in the request. This means the password information is only present in the request (in encoded form of course) from the application proxy to the WebSphere CloudBurst Appliance.
The good news about the application-based proxy approach is that it is simple to put in place. I composed one using the open source Apache Wink project. The Apache Wink project is an open source implementation of the JAX-RS specification (and then some), and it enables you to develop POJOs that are in turn exposed in a RESTful manner. In my case, I had a single resource POJO:
The Apache Wink runtime routes any HTTP GET request whose path is like /resources/* to the getResources method in the WCAResource class. This method passes along information taken from the query string (the host name of the target WebSphere CloudBurst Appliance and the requesting WebSphere CloudBurst username), as well as the HTTP path information and sends it on to the getResource method declared as follows:
The getResource method above uses the WebSphere CloudBurst host name and the request path to construct the URL for the corresponding WebSphere CloudBurst REST API call. Next, it constructs an Apache Wink Resource object and sends the REST request along to the WebSphere CloudBurst Appliance. How do we authenticate this request? We use the WebSphere CloudBurst username (sent as a query string parameter) to retrieve the appropriate encoded password information. Once we have that, we construct the necessary header for basic authorization over SSL.
The application-based proxy shown here is simply a pass-through. It does not manipulate the data returned from the WebSphere CloudBurst REST API, nor does it map a single client-side call to multiple REST requests. However, it would be simple enough to extend it to do any of those things. If you have any questions about the code here, please let me know. I'd be happy to share more of the code, or talk about how and where to extend it.