Though I feel like we've come a long way in some of the initial confusion surrounding IBM CloudBurst and WebSphere CloudBurst, I still get quite a few basic questions on the solutions. The two most common questions are, 'Are they different products?', and 'Can/should I use them together?'. I put together a really brief overview that answers these questions and talks about the basics of the combined solution. I hope it provides a good introduction!
In previous posts, I have discussed the integration capability between WebSphere CloudBurst and Tivoli Service Automation Manager. Most recently, I discussed this in the context of integrating WebSphere and IBM CloudBurst. Today, I am happy to announce the publication of an article I co-wrote with Marcin Malawski from TSAM development on the subject of this integration.
If you are a WebSphere user interested in a holistic approach in building out a private cloud, I strongly recommend that you check the article out. If you are currently an IBM CloudBurst, IBM Service Delivery Manager, or Tivoli Service Automation Manager user and you provision a significant number of WebSphere environments, I strongly recommend that you check the article out. In fact, regardless of your current situation, do me a favor and check the article out!
As always, I look forward to feedback and comments. Good, bad, or indifferent. You can leave your comments here or on the article page. I look forward to hearing from you!
When I talk with WebSphere CloudBurst users, the topic of custom virtual images comes up frequently. In some cases they simply want to customize a shipped IBM Hypervisor Edition, and in other cases they want to create a completely custom image. Creating a customized version of an IBM Hypervisor Edition is relatively easy since we give you extend & capture in WebSphere CloudBurst. Creating a completely custom image has historically been a bit tougher, mostly owing ot the fact that there was not a standard tool or process for image assembly. I am happy to say that today's publication of the IBM Image Construction and Composition Tool changes all that.
Watch a demo of the IBM Image Construction and Composition Tool
The primary purpose of the Image Construction and Composition Tool is to enable a modular approach to virtual image construction, while taking into account the typical division of responsibilities within an organization. The tool allows the right people within an organization to contribute their specialized knowledge as appropriate to the virtual image creation process. This means OS teams can handle the OS and software teams can handle the appropriate software. A separate image builder can then use both OS and software components to meet the needs of users within the organization. Best of all, the image builder does not need intimate knowledge of how to install or configure any of the components in the image. They simply need to know which OS and software components to use.
When using the Image Construction and Composition Tool, you start by defining the base operating system you wish to use for your images. You can do this by importing an existing virtual image with an OS already installed, providing an ISO for the OS, or pointing to a base OS image on the IBM Cloud. The bottom line is that you have necessary flexibility to start with your certified or ‘golden’ operating system build. Once you have the base OS image defined in the Image Construction and Composition Tool, you can start defining custom software for use in the images you will compose.
In the tool, bundles represent the software you wish to install within a virtual image. The definition of a bundle contains two major parts: Installation and Configuration. The installation component of a bundle tells the Image Construction and Composition Tool how to install your software into the virtual image. You provide a script or set of scripts that install the necessary components into your image, and you direct the tool to call these scripts. These tasks run once during the initial creation of the virtual image, thus allowing you to capture large binaries, long-running installation tasks, or other necessary actions directly into your image.
The configuration section of a bundle defines actions that configure the software installed into the image. Like with the installation tasks, you provide a script or set of scripts for configuration tasks. Unlike installation tasks that run exactly once, configuration scripts become part of the image’s activation framework and as such, run during each image deployment. Using the tool, you can define input parameters for configuration scripts and optionally expose them so that users can provide values for the parameters at image deploy-time. Configuration tasks are important in providing flexibility that allows users to leverage a single virtual image for a number of different deployment scenarios.
Once you have your base OS image and one or more bundles defined in the Image Construction and Composition Tool, you can compose a virtual image. To compose a virtual image, you extend the base OS image and add any number of bundles into the new image. A base OS image plus a set of bundles defines a unique image.
After you define the image you want to construct, you initiate a synchronize action in the Image Construction and Composition Tool. When you start the synchronize action, the tool first creates a virtual machine in either a VMware or IBM Cloud environment (based on how you configured the tool). Next, the installation tasks of each bundle you included in the virtual image run to install the required software. Finally, the tool copies the configuration scripts from each bundle into the virtual machine and adds them to the image’s activation framework. This ensures the automatic invocation of all configuration scripts during subsequent image deployments.
Once the image is in the synchronized state, you can capture it. Capturing the image results in the creation of a virtual image based on the state of the synchronized virtual machine. The tool also automates the generation of metadata that becomes part of the virtual image package. When the capture of the virtual image completes, you can export it from the Image Construction and Composition Tool and deploy it using WebSphere CloudBurst, Tivoli Provisioning Manager, or the IBM Cloud.
I am excited for users to get their hands on the Image Construction and Composition Tool. I believe it represents the first big step in helping users to design and construct more sustainable virtual images. Did I mention it is completely free to download and use? Visit the Image Construction and Composition Tool website for more details and a download link. I look forward to your comments and feedback.
In last week's post, I put the spotlight on various aspects of bundles in the Image Construction and Composition Tool. I finished with a look at a WebSphere CloudBurst virtual image created from the bundle. However, you do not just magically go from a bundle to an image that you can use in WebSphere CloudBurst, Tivoli Provisioning Manager, or on the IBM Cloud. Today, I want to show you how to go from a bundle to a custom virtual image using the IBM Image Construction and Composition Tool.
Once you have defined at least one bundle and one base operating system image, you are ready to compose a custom image. We already talked about creating a bundle, but the base operating system image is a new topic. You can do this by either starting from ISO and kickstart configuration files, or you can import an existing Open Virtual Appliance (OVA) image that contains your operating system of choice. Once you have that base image imported or defined in the Image Construction and Composition Tool, you can extend it to create a custom image on top of the base OS image.
After creating your extended image, you can add bundles that represent the software you want to install in your custom image. Simply click on the Software tab of the new virtual image. Click the add icon, and select the bundle that you want to add. You can add as many bundles as you would like to your custom image.
After adding a bundle, it will show up in the Planned list of software for the image. Click on it to display its details in the right side of the screen. You will notice General, Install, and Configuration sections for the bundle. In the Install section, you will find a list of the installation parameters you defined for the bundle. You can provide values for the parameters at this time.
If you click on the Configure section, you will see all of the configuration paramters you specified for the bundle. You can provide default values, and you can specify whether or not these should be configurable by deployers of your custom image. If you mark them as configurable, users will be able to provide values for the parameters at image deploy time, regardless of whether they provision the image using WebSphere CloudBurst, Tivoli Provisioning Manager, or the IBM Cloud.
After you add the necessary bundles and specify installation and configuration data, you can save the image. Upon saving, the image status changes from Synchronized to Out of Sync.
Now you are ready to synchronize the image. To do this, simply click the synchronize icon. This will result in the creation of a virtual machine in the cloud envrionment (VMware or IBM Cloud) you defined in the selected cloud provider. The Image Construction and Composition Tool will then invoke the appropriate installation tasks (per the bundles you included in the image) within the running virtual machine. It will also copy over any configuration scripts you defined in the bundle.
After a while, the synchronization process completes, and the image returns to the Synchronized state. At this point, you are ready to capture the image by clicking the capture icon. This results in the creation of an OVA virtual image with your customizations. When the capture process completes, the image status changes to Deployable.
Once the image is in the deployable state, it is nearly ready to use. If you are using the IBM Cloud as your cloud provider, you can simply mark the image complete by clicking the complete icon. At this point, the image will show up in your private catalog on the IBM Cloud and it is ready to use. If you are using VMware as the cloud provider, you need to export the image. Click the export icon and provide information about an SCP-enabled server to which you want to export the image. Ideally, this location is directly reachable by the WebSphere CloudBurst or Tivoli Provisioning Manager environment into which you will import the image.
You can monitor the export status in a separate window by clicking on a link shown after clicking the OK button in the dialog above. When the export finishes, you are ready to import your new custom virtual image into WebSphere CloudBurst or Tivoli Provisioning Manager.
I hope the last three posts have given you a better idea of what the new IBM Image Construction and Composition Tool is all about. There will definitely be more to come about this tool in the near future, but in the meantime, if you have any questions or comments, please reach out to me. Until then, good luck and full speed ahead on your custom image compositions!
I hardly ever have a conversation about WebSphere CloudBurst, or generally cloud computing for application middleware, without the topic of databases coming up. Databases are such an important piece of nearly every application middleware environment, so users want to be sure that whatever they do for their application servers, they can also do for the databases on which their applications rely. That is why the capability to deploy DB2 from WebSphere CloudBurst has been around for as nearly as long as the capability to deploy WebSphere Application Server.
Even though DB2 deployment capability has been around for a while, there are still some common misconceptions regarding the offering. First, I have talked to a fair number of users who are under the impression that we only offer a trial version of DB2 for deployment via WebSphere CloudBurst. While that was true for the first few months of the offering, that is no longer the case. For several months now, a fully supported, 64 bit, production-ready DB2 image has been ready for use in WebSphere CloudBurst. If you were waiting for a DB2 image that you could go live with, wait no longer!
The other misconception, or rather, point of confusion, arises from the fact that the DB2 image for WebSphere CloudBurst is not, by name, a Hypervisor Edition image. I can assure you that is in name only. The DB2 image looks like and behaves like any other IBM Hypervisor Edition image once you load it into the appliance. You can use it to build and deploy patterns in the same way you use other images in WebSphere CloudBurst. You may just have trouble finding it if you search for 'DB2 Hypervisor Edition' as opposed to 'DB2 Server for WebSphere CloudBurst Appliance.'
Instead of going into further detail, I want to refer you to a blog entry from a fellow IBMer, Leon Katsnelson. Leon is a program director for DB2 and is responsible for the team that develops and delivers the DB2 image for WebSphere CloudBurst. In his most recent post, he provides a nice overview of the image and gives good information for those looking to use DB2 and WebSphere CloudBurst (there is also a bit on cloud computing at the beginning that I think is spot on). Check out Leon's post, and let us know what you think!
One of the key benefits of WebSphere CloudBurst adoption is rapid -- seriously fast -- deployments of middleware application environments. Our users are leveraging the appliance to bring up enterprise-class middleware environments in mere minutes. If you know a little bit about WebSphere CloudBurst, that statistic may be a little surprising considering the appliance dispenses large virtual images from the appliance over the network to a farm of hypervisors. You may ask how the appliance can achieve such rapid deployments in light of the mere physics involved in transferring large amounts of data over a network. The simple answer is caching of course!
WebSphere CloudBurst creates a cache for each unique virtual image on datastores associated with the hypervisors in your cloud. On subsequent deployments of the same virtual image to the same datastore, WebSphere CloudBurst does not need to transfer the image over the wire. It simply uses the virtual disks that are in the cache on the datastore. In the context of the virtual image cache, the deployment process goes something like this:
WebSphere CloudBurst identifies the images necessary to deploy the pattern selected by the user.
WebSphere CloudBurst identifies the hypervisors and associated datastores that will host the virtual machines created during deployment.
WebSphere CloudBurst checks the selected datastores to see if they already have caches for the images it will be deploying. From here, one of two things happens:
WebSphere CloudBurst detects that there is no cache on the datastore and transfers the images over to the hypervisor, thereby creating the cache on the underlying datastore.
WebSphere CloudBurst detects that there is a cache on the selected datastore and uses that cache in lieu of transferring the disk over the wire.
The process may sound complicated, but it is completely hidden from you, the user. You do not need to know how the cache works since WebSphere CloudBurst handles all of these interactions. So, why am I telling you all of this then? As a WebSphere CloudBurst user, it is good to be aware of the cache for two main reasons. First, you need to account for the storage space the cache needs when doing capacity planning for your WebSphere CloudBurst cloud. Second, anytime you upload or create a new image through extend and capture, I would strongly suggest you automatically prime the cache for this new image. You can do this by simply deploying a pattern built on the image to each unique hypervisor/datastore in your environment. This may take a temporary re-arrangement of cloud groups, but it is a simple process, and it guarantees rapid deployments for all users of the new image.
I hope this sheds a little light on a subject we do not discuss too often. As always, if you have any questions, do not hesitate to let me know!
I hate sitting on secrets. I always have. I understand that sometimes it's in the best interest of everyone (and your job) to keep tight lips, but that does not make it any more fun. Inevitably, the run-up to our annual Impact conference means everyone in the lab is doing their fair share of secret keeping -- just waiting for announce time. For a lot of us, that day ended Tuesday with the announcement of the IBM Workload Deployer v3.0.
Now, you may be wondering, 'I have never heard of this. Why is it version 3.0??' Well, IBM Workload Deployer is a sort of evolution of the WebSphere CloudBurst Appliance, which was previously at version 2.0. This is good news for all of our current WebSphere CloudBurst users because all of the functionality (plus new bits of course) that they have been using in WebSphere CloudBurst are present in IBM Workload Deployer. You can use and customize our IBM Hypervisor Edition images in IBM Workload Deployer. You can build and deploy custom patterns that contain custom scripts in order to create highly customized IBM middleware environments. So, what's the big deal here? Two words: workload patterns.
Workload patterns represent a new cloud deployment model and are an evolution of the traditional topology patterns you may have seen with WebSphere CloudBurst Appliance (I am a little torn between evolution and revolution, but that's splitting hairs). Fundamentally, workload patterns raise the level of abstraction one notch higher than topology patterns and put the focus on the application. That means, when you use a workload pattern the focus is on the application instead of the application infrastructure. Perhaps an example would be helpful to illustrate how a workload pattern may work in IBM Workload Deployer.
Let's consider the use of a workload pattern that was part of the recent announcement, the IBM Workload Deployer Pattern for Web Applications v1.0. Just how might something like this work? It's simple really. You upload your application (maybe a WAR or EAR file), upload a database schema file (if you want to deploy a database with the solution), upload an LDIF file (if you want to setup an LDAP in the deployment to configure application security), attach policies that describe application requirements (autonomic scaling behavior, availability guidelines, etc), and hit the deploy button. IBM Workload Deployer handles setting up the necessary application middleware, installing and configuring applications, and then managing the resultant runtime in accordance with the policies you defined. In short, workload patterns provide a completely application centric approach to deploying environments to the cloud.
Now, if you are a middleware administrator, application developer, or just a keen observer, you probably have picked up on the fact that in order to deliver something as consumable and easy to use as what I described above, one must make a certain number of assumptions. You are right. Workload patterns encapsulate the installation, configuration, and integration of middleware, as well as the installation and configuration of applications that run on that middleware. Most of this is completely hidden from you, the user. This means you have less control over configuration and integration, but you also have significantly reduced labor and increased freedom/agility. You can concentrate on the development of the application and its components and let IBM Workload Deployer create and manage the infrastructure that services that application.
Having shown and lobbied a bit for the benefits of workload patterns, I also completely understand that sometimes you just need more control. That is not a problem in IBM Workload Deployer because as I said before, you can still create custom patterns, with custom scripts based on custom IBM Hypervisor Edition images. The bottom line is that the IBM Workload Deployer offers choice and flexibility. If your application profile meshes well with a workload pattern, by all means use it. If you need more control over configuration or more highly customized environments, look into IBM Hypervisor Edition images and topology patterns. They are both present in IBM Workload Deployer, and the choice is yours.
If you happen to be coming to IBM Impact next week, there will be much more information about IBM Workload Deployer. I encourage you to drop-by our sessions, ask questions, and take the opportunity to meet some of our IBM lab experts. Hope to see you in Las Vegas!
IBM Impact 2011 was a wildly busy week! Customer meetings, entertaining keynotes, informative sessions, and hands-on labs packed the 6 days with more than enough action. I spent a lot of the week presenting sessions and conducting labs for the newly announced IBM Workload Deployer. As one would expect with any new announcement, we got tons of questions about IBM Workload Deployer. While I cannot capture all the questions and their answers here, I will try to cover some of the more prevalent ones below.
Question: What happened to WebSphere CloudBurst?
Answer: The short answer is, it simply went through a rename. WebSphere CloudBurst became IBM Workload Deployer v3.0. The version 3.0 acknowledges this is an evolution of what we started with WebSphere CloudBurst, which was at version 2.0. Why remove WebSphere from the name? The fact that this is now an IBM branded offering is more accurate as it is capable of deploying and managing more than just WebSphere software.
Question: What is new in IBM Workload Deployer?
Answer: While there are many new features that I will be talking about over the coming months, the most prominent new facet is the introduction of workload patterns (also referred to as virtual application patterns). As opposed to topology patterns (traditionally referred to as simply patterns in the WebSphere CloudBurst product), workload patterns raise the level of abstraction to the application level. Instead of focusing on application infrastructure and its configuration as you do with topology patterns, workload patterns allow you to focus on the application and its requirements. When using workload patterns, you provide the application, attach policies that specify functional and non-functional requirements, and deploy. IBM Workload Deployer handles deploying and integration the middleware infrastructure necessary to support the application, and it automatically deploys your application on top of that middleware. In addition, IBM Workload Deployer manages the application runtime in accordance with the policies that you specify in order to provide capabilities such as runtime elasticity.
Question: If I am a current WebSphere CloudBurst user, what does this mean for me?
Answer: Not to worry. You will be able to use all of your WebSphere CloudBurst assets (patterns, scripts, images) in the new IBM Workload Deployer. All of the capabilities previously in WebSphere CloudBurst are present in IBM Workload Deployer (terminology may vary slightly -- topology pattern instead of just pattern for instance). Additionally, we continue to expand on the functionality that you are familiar with from WebSphere CloudBurst. This includes updates for Environment Profiles, new IBM Hypervisor Edition images, new pattern building capabilities, and more. Stay tuned for more information about these new features and for information on how you can move your WebSphere CloudBurst resources to the new IBM Workload Deployer.
Question: How do I choose between using workload and topology patterns?
Answer: There are a number of factors that will lead you to using either workload patterns, topology patterns, or both. The primary decision point will be how much control you really need (not want). When using workload patterns, you sacrifice some customization control over the configuration, integration, and administration of the middleware application environment since the workload pattern and management model abstracts away the 'guts' of the system. Everything about the workload pattern is application-centric. On the other hand, topology patterns give you intimate control over the configuration, integration, and administration of the middleware application environment. As a general rule of thumb, if your application requirements match the capabilities of a workload pattern, that is the way to go as it can greatly reduce complexity and cost associated with deployment and management. If a workload pattern does not meet the needs of your application, topology patterns can still greatly reduce cost and complexity and you can tailor them to fit almost any need. Beyond generalities, there is no hard and fast rule for choosing one over the other. It comes down to understanding your application environment and its needs.
Question: Is IBM Workload Deployer an appliance like WebSphere CloudBurst?
Answer: Yes, it is still an appliance, but an updated one! The new appliance is 2U, and it provides more storage, processing power, and memory. It is still just as easy to setup, but just slightly bigger.
Well, that is all for now, but I will be back many times over the coming months with more information. In the meantime, if you have any questions, please leave them in a comment below.
Jason McGee will be leading the second GWC Lab Chat this week on Wednesday, 4/20. The very timely topic is related to recent announcements from IBM regarding the IBM Workload Deployer (see previous posts). Entitled "Application-Centric Cloud Computing" the discussion will focus on the concept of deploying and managing your application workloads in a shared, self-managed environment rather than manually creating and managing the application middleware topologies. It places the focus on the application rather than the infrastructure. This concept promises to deliver greater simplicity, elasticity, and
density among other things. It can position your business to react more
quickly and efficiently to the increasing demands of your customers and
free you from the managing all of the details.
Many of you may have already heard Jason speak last week at IMPACT 2011 in the cloud mini main tent or perhaps at any number of other sessions that Jason was involved in. Jason is the key architect behind IBM's WebSphere cloud activities. Obviously, Jason understands the cloud space very well and has a clear view of the evolution into Application-Centric Cloud Computing. This GWC Lab Chat will provide the opportunity to get your questions answered and share your perspective on this technology.
Jason will provide a brief introduction to the concepts and ideas and then lead an open discussion. Put it on your calendar and plan to attend - and please plan to bring your questions and comments to help foster a rich discussion. We want to hear from you.
If you haven't registered yet it is not too late - learn more and register here. It is easy to register and there is no cost. This is a very timely event and a great way to dig a little more deeply into concepts you first heard at IMPACT or perhaps hear them for the first time. Don't miss it!
When one uses IBM Workload Deployer (previously WebSphere CloudBurst) to deploy a virtual system pattern, they benefit from a completely automated deployment process. The automation includes the creation and placement of virtual machines, injection of IP addresses, initiation of internal processes, and invocation of included scripts. Most of these processes are straightforward and require little more than a brief overview. However, the placement of virtual machines stands out, and it's inner workings are the subject of quite a few questions when I discuss the appliance. With that in mind, I thought I would provide a little more information on how the placement algorithm in IBM Workload Deployer works.
The placement subsystem in IBM Workload Deployer considers three primary elements: compute resource, availability, and license optimization. Compute resource availability is the gating factor for placement. That means that IBM Workload Deployer first looks at the available CPU, memory, and storage resource in the collection of hypervisors making up the cloud group(s) you are targeting for deployment. If a particular hypervisor cannot provide enough resource based on the amount you requested for your deployment, then it is automatically taken out of the eligible hosts pool. It is important to note that IBM Workload Deployer will overcommit CPU, and it will overcommit storage if you direct it to do so. It will not overcommit memory because that could severely degrade the performance of the application(s) running in the virtual machines.
After choosing the pool of hypervisors that are capable of hosting the virtual machines in your deployment from a compute resource perspective, the appliance then considers high availability. To better understand this particular placement stage, let's consider an example. Consider you are deploying a pattern based on WebSphere Application Server Hypervisor Edition and it contains two custom node parts. It is conceivable, and in fact likely, that these two custom node parts will host members of the same cluster, and thus the two nodes will support the same applications. As such, IBM Workload Deployer will attempt to place the two custom nodes on different physical machines in order to prevent a single point of failure. Of course, this depends on having two hypervisors with enough resource (CPU, memory, storage) to host the virtual machines, but the appliance makes that decision in the first placement stage.
After considering compute resource and high availability, IBM Workload Deployer moves to the last stage of placement: license optimization. In this stage, the placement subsystem attempts to place the virtual machines on hypervisors in a way that minimizes the licensing cost to you. The appliance can do this because it is aware of IBM virtualization licensing rules and takes those into account during this stage (if you aren't familiar with virtualization licensing rules and you are curious, ask you're sales representative to explain some time). During this stage, it will not violate any resource overcommit directives or rules in place, nor will it compromise system availability, but it will seek to minimize costs within these parameters.
At this point, I should make something clear that may already have occurred to you. You can override most of these placement rules by creating a cloud group containing only one hypervisor. In this case, IBM Workload Deployer will put all virtual machines on the single hypervisor until it runs out of compute resource (memory is likely to be the constraining factor). I would not suggest that you do this unless you have a good reason or you are in a simple pilot phase, but I do like to point out the art of the possible!
While not incredibly deep from a technical perspective, I do hope that this provided a few helpful details on what goes on during the placement stages of deployment. If you have any questions, do let me know.
Application-centric cloud computing is the main thrust behind the new capabilities of IBM Workload Deployer v3.0. But what does that really mean? After all, application-centricity is really just a concept. Granted, it is an important concept, but it is fairly meaningless until it is put into action or implemented. IBM Workload Deployer does just that with its new Virtual Application Patterns (VAPs).
VAPs are the embodiment of the workload pattern approach I briefly discussed in an overview post a few weeks back. The idea with a VAP is to give the user an interface through which they can provide their application, specify dependencies, declare functional and non-functional requirements and then deploy. Of course application middleware is a part of the overall solution, but IBM Workload Deployer has the smarts to build, configure, and integrate the necessary infrastructure in order to support the user's application. This is completely hidden from the user, so they are liberated to focus on the application and its requirements.
If we scratch a bit further beneath the surface of a VAP, we see that these patterns contain three primary pieces. These primary pieces are components, links, and policies, and they are fundamental to understanding how virtual application patterns work. Let's start with the building blocks of VAPs, components. Put simply, components represent different resources and functionality profiles that make up your application environment. As an example, the IBM Workload Deployer Pattern for Web Applications is a VAP that contains components for an EAR file, WAR file, message queue, and any number of other components that are typical requirements for a web application. The components will certainly vary based on the workload type (i.e. the components included in a web application VAP would be different than those included in a batch application VAP), but they are the foundation of any VAP.
From the ground up, the next logical element we come to in the VAP is a link. A link is a way to declare a dependency or integration point between two components. As an example, consider a VAP with a WAR file component and a database component. You might draw a link between the WAR component and the database component to indicate that your web application uses or otherwise depends on the database. IBM Workload Deployer interprets this link, and takes it as a directive to configure the integration between the two components as a part of deployment. In this case, that may mean configuring a data source in the application's container. This is just a simple example, and an application may have any number of links between components.
Finally, we come to the policy element within the VAP. A policy is a way for a user to specify functional and non-functional requirements for their application environment. Users attach policies to the VAP, or to components in their VAP, and IBM Workload Deployer interprets and enforces those policies. In the context of a web application, one example of a policy could be a scaling policy. The scaling policy might indicate scaling requirements for the application that included minimum application instances, maximum application instances, and conditions that triggered scaling activities. IBM Workload Deployer would use the information in a scaling policy within a VAP to appropriately manage the deployed, running environment. Other examples of a scaling policy may include a JVM policy that provides configuration directives for the java virtual machines in your application environment or a logging policy that defines logging configuration options. In any case, the policy element allows VAP builders to influence the configuration and management of the application environment.
In the example VAP below you can see the use of components (Enterprise Application, Database, User Registry, Messaging Service), links (blue lines between components), and policies (Scaling Policy, JVM Policy):
In total, when I look at a VAP a particular word sticks out to me: declarative. VAPs really enable declarative, application-centric cloud computing. What do I mean? By declarative, I mean you are telling IBM Workload Deployer what you want, but not necessarily how you want it done. It is the job of IBM Workload Deployer to take care of the how. This shift in approach to application environments enables the potential for significant savings, and more importantly to me, lays the foundation for a more agile, flexible approach to deploying and managing application environments.
There will be more in the weeks and months to come on IBM Workload Deployer, so stay tuned. I also want to put a plug in for a new blog from Jason McGee. For those that do not know Jason, he is an IBM Distinguished Engineer, and the lead architect behind IBM Workload Deployer. Be sure to check out his blog for insights on this new offering, as well as for all things cloud.
One of the fundamental tenants of IBM Workload Deployer is a choice of cloud deployment models. Starting in v3.0, users will be able to deploy to the cloud using virtual appliances (OVA files), virtual system patterns, or virtual application patterns. The ability to provision plain virtual appliances is a way to rapidly bring your own images, as they currently exist, into the provisioning realm of the appliance. As such, I think the use cases and basis for deciding to use this deployment model are fairly evident. However, when comparing the two patterns-based approaches, virtual system patterns and virtual application patterns, the decision requires a bit more scrutiny.
Our pattern approach is a good thing for you, the user. Basically, when we refer to patterns in the context of cloud, we are referring to the encapsulation of installation, configuration, and integration activities that make deploying and managing environments in a cloud much easier. Regardless of what kind of pattern you end up using, you benefit from treating a potentially complex middleware infrastructure environment or middleware application as a single atomic unit throughout its lifecycle (creation, deployment, and management). In turn, you benefit from decreased costs (administrative and operational) and increased agility via rapid, meaningful deployments of your environments. That said, it is imperative to understand the differences between virtual system and virtual application patterns, and more importantly, it is important to understand what those differences mean to you. Let's start by considering the admittedly simple 'Cloud Tradeoff' continuum below.
In the above graph, the X-axis represents the degree to which you have customization control over the resultant environment. The degree of control gets lower as we move from left to right. The left Y-axis represents total cost of ownership (TCO), which decreases as we move up the axis. The right Y-axis represents time to value, which similarly decreases as we go up the axis. Naturally, enterprises want to move up the Y-axis, but, and it can be quite a big but, they are sometimes hesitant to relinquish much control (move to the right on the X-axis) in order to do so. In that light, I think it helps to explore our two patterns-based approaches a bit more.
The most important thing to understand about this continuum is that the X-axis really represents the customization control ability from the point of view of the deployer and consumer of the environment. An example is probably the best way to explain. Let's consider a fairly simple web service application that we want to deploy to the cloud. If we were to use a virtual system pattern to achieve this, we would probably start by using parts from the WebSphere Application Server Hypervisor Edition image to layout our topology. We may have a deployment manager, two custom nodes, and a web server. After establishing the topology, we would add custom script packages to install the web service application and then configure any resources the application depended on. Users that wanted to deploy the virtual system pattern would access it, provide configuration details such as the WAS cell name, node names, virtual resource allocation, and custom script parameters, and then deploy. Once deployed, users could access the environment and middleware infrastructure as they always have. That means they could run administrative scripts, access the administrative console provided by the deployed middleware software, and any other thing one would normally do. The difference in using virtual system patterns is not necessarily the operational model for deployed environments (though IBM Workload Deployer makes some things, like patching environments, much easier). Instead, the difference is primarily in the delivery model for these environments.
Using a virtual application pattern to support the same web service application results in a markedly different experience from both a deployment and management standpoint. In using this approach, a user would start by selecting a suitable virtual application pattern based on the application type. This may be one shipped by IBM, such as the IBM Workload Deployer Pattern for Web Applications, or it may be one created by the user through the extensibility mechanisms built into the appliance. After selecting the appropriate pattern, a user would supply the web service application, define functional and non-functional requirements for the application via policies, and then deploy. The virtual application pattern and IBM Workload Deployer provide the knowledge necessary to install, configure, and integrate the middleware infrastructure and the application itself. Once deployed, a user manages the resultant application environment through a radically simplified lens provided by IBM Workload Deployer. It provides monitoring and ongoing management of the environment in a context appropriate for the application. This means that there are typically no administrative consoles (as in the case of the virtual application pattern IBM ships), and users can only alter well-defined facets of the environment. It is a substantial shift in the mindset of deploying and managing middleware applications.
Okay, with that explanation in the bag, let's revisit the diagram I inserted above. I hope it's clear that, all things being equal, virtual application patterns indeed provide the lowest TCO and shortest TTV because of the degree to which they encapsulate the steps involved in setting up complex middleware application environments. So, let's get back to my assertion that the customization control continuum really applies to the deployer and consumer. Why do I say that? It's simple. In the case of either the virtual system pattern or the virtual application pattern, the pattern composer has quite a bit of liberty in how they construct things. Sure, we enable you right out of the chute by shipping pre-built, pre-configured IBM Hypervisor Edition images, as well as pre-built virtual system and virtual application patterns. The key is though, that the IBM Workload Deployer's design and architecture also enables you to build your own patterns -- be they the virtual system or virtual application type. With anywhere from a little to a lot of work, you can build virtual system and virtual application patterns tailored to your use cases and needs.
At this point, you may be saying, "Well now you have really confused things! How am I supposed to decide what kind of patterns-based approach fits my needs?" I have some advice in that regard. First, map your needs to things that we enable with the assets you get right out of the box with IBM Workload Deployer. If your application fits into the functional scope of one of the virtual application patterns that we ship, use it. If you can support the application by using IBM Hypervisor Edition images, virtual system patterns, and custom scripts, do it. In this way, you benefit most from the value offered by IBM Workload Deployer. However, if you find that you cannot use any of the assets we provide right out of the box (e.g. you want to deploy your environment on software not offered in IBM Hypervisor Edition form or in a virtual application pattern), then ask yourself one simple question: "What do I want my user's experience to be?"
In this sense, I primarily mean a user to be a deployer or consumer of your patterns. You need to decide whether you favor the middleware infrastructure centric approach afforded by virtual system patterns, or if you prefer the application centric approach proffered by virtual application patterns. There is no way to answer this generically for all potential IBM Workload Deployer users. Instead, you have to look at your use case, understand what's available to help you accomplish that use case, and finally, decide on what you want your user's experience to be. I hope this helps!
WebSphere configuration management practices are common items of conversation that comes up when I am talking with users about IBM Workload Deployer (formerly WebSphere CloudBurst). This conversation can take on so many different avenues that it is hard to capture all of them in a short blog post. So, for the sake of this post, let's consider two facets of WebSphere configuration management. The first facet is addressing the need to consistently arrive at the same configuration across multiple deployments of a given WebSphere environment. The second facet involves managing the configuration of a deployed environment over time to protect against living drift. What is the best way to tackle these two challenges? Well, it comes down to picking the right tool for the job.
When it comes to ensuring consistency of initial WebSphere configuration from deployment to deployment, there is really no better means than patterns-based deployments enabled by IBM Workload Deployer. Whether you are using a virtual system or virtual application pattern, the bottom line is that you are representing your middleware application environments as a single, directly deployable unit. When you need to stand that environment up, you simply deploy the pattern. The deployment encapsulates the installation, configuration, and integration of the environment, and your applications if you so choose. The benefit of this approach is that once you get your pattern nailed down, you can be extremely confident that the initial configuration of your environments is extremely consistent from deploy to deploy. Basically, no more bad deployments because someone forgot to run configuration step 33 out of 100!
Because we talk about the benefits of consistency provided by our IBM Workload Deployer patterns, users often ask what IBM Workload Deployer does in terms of configuration governance for deployed environments. In other words, they ask how IBM Workload Deployer helps them to track configuration changes or compare the configuration of a deployed environment to a known good one. The honest answer is that this is a bit beyond the functional domain of the appliance. While IBM Workload Deployer does allow you to manage the deployed environment (apply fixes, update deployed applications, snapshot, etc.), it does not layer some of the common configuration governance concerns on top of that. However, there is a good reason why the appliance does not focus on that. It's because Rational Automation Framework for WebSphere does!
If you find yourself wanting to actively track configuration changes, periodically (and automatically at specified intervals) compare configuration changes to a 'golden' baseline, import configurations of a known good environment, apply common configuration across a number of cells, then the capabilities of RAFW would likely be of interest to you. It can do all this and give you an incredible toolbox of out-of-the-box application deployment and configuration capabilities for WebSphere environments. In my mind, for those that spend a good deal of time dealing with WebSphere configuration, whether it be deploying applications, configuring containers, or debugging inadvertent changes, an examination of RAFW functionality is a must.
Now it is time for a bit of disclaimer/clarification. I am not suggesting that you pick one or the other when it comes to IBM Workload Deployer and RAFW. In fact, there are many scenarios where 1+1=3 with these two solutions, and I have written about it many, many times (including this article). That said, I think it is important to highlight the relative strengths of each product, so that it is easier to map it back to your pain points. In honesty, many of the users I talk with have challenges in getting the initial configuration right AND managing it over time. That kind of problem beckons for the integrated IBM Workload Deployer/RAFW solution.
Of course, technology only gets you so far when it comes to these kinds of problems. It would be disingenuous of me to suggest otherwise. It has always been and will continue to be important to establish clear and rigorous processes around the way you deploy, manage, and change environments. This just gives you an idea of some of the tools you can leverage to aid in the implementation of those processes.
More and more, I am getting a question about how to bring existing WebSphere environments into IBM Workload Deployer. While "bringing in an environment" can mean any number of things, let's take it to mean that a user wants to import their existing WebSphere cells, applications, and configuration into IBM Workload Deployer as a pattern they can subsequently deploy. While there may not be a big red easy button in the appliance that lets you point to an existing environment and import it, there are a couple of techniques that one can employ. I have covered both techniques before, but since I'm getting the question with increasing frequency, I felt like it was time for recap.
The first option is to use a combination of IBM Workload Deployer and Rational Automation Framework for WebSphere. This is a use case I have spoken about numerous times at conferences and in blog posts and articles. In fact, you can read a little about it here. In this sense, RAFW provides excellent capabilities to point at an existing cell, and import everything about it. This includes WebSphere configuration, applications, shared libraries, and more. Once imported as a RAFW project, you can use the IBM Workload Deployer integration script package provided by RAFW to replay that configuration on top of deployments created by the appliance.
The second option is something I talk about a little less frequently. This option revolves around the use of a sample script (provided for free in our samples gallery) that you can run against existing WebSphere cells. The invocation of this script produces IBM Workload Deployer script packages that you can use in patterns to apply the configuration of the target cell to your new cloud-based deployments. Under the covers the utility script and resultant script packages use backupConfig and restoreConfig respectively. They do ensure the update of the cell, node, and host names during the restoreConfig execution (which happens automatically during pattern deployment). Beyond that, the use of the script is subject to the same limitations and rules in place for the use of the backupConfig and restoreConfig commands. You can read more about this capability, watch it in action, and download it for free.
I hope this is all useful information for those of you looking for ways to import existing environments into IBM Workload Deployer as patterns. If you have any questions, please let me know!
As I have mentioned before, IBM Workload Deployer v3.0 introduces choices in pattern-based deployment models. One of those models, virtual system patterns, is a carry over from the WebSphere CloudBurst Appliance. When you use virtual system patterns in IBM Workload Deployer, you can take advantage of all of the techniques you put to use in WebSphere CloudBurst. This is certainly good news for current WebSphere CloudBurst users, but it goes a bit further. Instead of simply maintaining the status quo with virtual system patterns, which would have been reasonable considering the introduction of virtual application patterns, we chose to continue to expand on your customization options for this pattern deployment model. In particular, I want to discuss three new features in IBM Workload Deployer that may help you to better construct and manage virtual system patterns.
The first new feature is one that I have been eagerly awaiting. In the new version of the appliance, we provide you with the ability to specify part and script package ordering in your pattern. This means that, within the virtual system pattern editor, you can tell IBM Workload Deployer in which order to start the virtual machines in your pattern, and you can specify in which order to invoke the script packages within the pattern during deployment. This eliminates the need for special script invocation orchestration logic in your pattern (I had customers resorting to a semaphore like approach using a shared file system), and it allows you to be more declarative about the virtual machine bring-up process. There are constraints, specifically with the part ordering. Some images will impose an implied part start-up order that you cannot change. For instance, deployment manager parts in the WebSphere Application Server Hypervisor Edition image must start before custom node parts. The good news is the pattern editor will not allow you to specify a part start-up order that violates these constraints. The image below shows an example of the ordering view in the virtual system pattern editor.
Another new feature that may influence the way you build virtual system patterns is the introduction of Add-Ons. You can think of Add-Ons as special script packages that you can include in your virtual system pattern that perform system-level configuration actions. Specifically, you can include add-ons in your virtual system pattern to add an operating system user, add a virtual disk, or add a NIC during the deployment process. You include Add-Ons in your pattern by simply dragging and dropping them onto a part in your pattern, just as you do with script packages today. The difference between script packages and Add-Ons is that IBM Workload Deployer will ensure the invocation of all Add-Ons before any other scripts run during deployment. We include default Add-On implementations for adding a user, disk, and NIC.
The last new feature I want to talk about today has more to do with how you manage or govern the deployment of virtual system patterns. In WebSphere CloudBurst 2.0, we introduced the idea of Environment Profiles as a way to extend your customization reach into the deployment process. Initially, these profiles gave you the ability to directly assign IP addresses to virtual machines in your deployment, declaratively specify virtual machine naming formats, and easily split a single pattern deployment across multiple cloud groups. In IBM Workload Deployer, you will be able to use these same profiles to set resource consumption limits for pattern deployments. In particular, you will be able to set cumulative limits for virtual CPU, memory, storage, and software licenses used by deployments tied to a specific profile, thereby giving you finer-grained control over cloud resource consumption. The picture below shows the new resource limit aspects of environment profiles.
Virtual system patterns are key in the deployment model choices for IBM Workload Deployer. Not only did we carry the concept over from WebSphere CloudBurst to IBM Workload Deployer, but we made it even better. Expect this trend to continue!
Among the major features of the new virtual application pattern in IBM Workload Deployer is the notion of elasticity. That is, as your application needs more resources, it gets them. When your application can meet its SLAs with fewer resources, the environment shrinks. With this kind of pattern, you enable elasticity by specifying a policy and defining the scaling trigger (i.e. CPU usage, application response times, database response times, etc.). What may have been a bit lost in some of these new announcements regarding IBM Workload Deployer is the fact that you can now leverage this core feature of cloud, elasticity, in your virtual system patterns.
If you have read this blog in the past, you probably already know that the Intelligent Management Pack is an option for virtual system patterns built using WebSphere Application Server Hypervisor Edition. When you enable the Intelligent Management Pack option, you are essentially building and deploying WebSphere Virtual Enterprise (WVE) environments. For those of you not familiar with WVE, the best way to describe it is that it provides you with application and application infrastructure virtualization capabilities. Of its many capabilities, one most germane to our discussion today is the ability for users to attach SLAs to applications and then have WVE automatically prioritize requests and manage resources in order to meet those SLAs. Inherent in this capability is the ability to dynamically start and stop application server processes (JVMs) as required. In other words, WVE provides JVM elasticity.
The fact that WVE provides JVM elasticity is nothing new. Further, IBM Workload Deployer started providing virtual machine (VM) elasticity in previous versions (when it was WebSphere CloudBurst). With this feature, you could add or remove VMs to an already deployed virtual system using dynamic virtual machine operations provided by the appliance. The catch was that the VM elasticity was a manual action and you could not link this elasticity to the same SLAs tied to your applications. Well, thanks to a new feature in WebSphere Virtual Enterprise and easy integration provided by the Intelligent Management Pack, this is no longer the case.
Starting in IBM Workload Deployer 3.0, you can take advantage of a new WVE feature called Elasticity Mode when using the Intelligent Management Pack. Elasticity mode is not unique to IBM Workload Deployer, but a concept new to the base WVE product. It allows one to define actions for how WVE should grow and shrink the set of nodes used by application server resources. Like the basic JVM elasticity capability in WVE, these node elasticity actions trigger based on SLAs tied to your applications. Consider the case that you are using elasticity mode and your application is not currently meetings its SLA. If WVE does not think it can start any more application server instances on the current set of nodes, it will grow the set of nodes per your elasticity configuration. Conversely, if WVE detects that it can meet SLAs with fewer nodes, it will shrink the resources per your elasticity configuration.
In IBM Workload Deployer, using elasticity mode becomes even easier. All you need to do is use the Intelligent Management Pack and enable the elasticity mode option in your virtual system patterns. When you do this, you get automatic integration between IBM Workload Deployer and the deployed WVE environment. What does that mean? It means that if WVE detects it needs more nodes, it will automatically call back into IBM Workload Deployer and request that the appliance provision a new VM that will serve as a node for application server processes. It also means that if WVE detects it could meet SLAs with fewer resources, it will call into IBM Workload Deployer and ask it to remove a node. All of this happens without any user scripting. All you have to do is enable this option in your patterns and configure SLAs appropriate for your applications.
To me, this exciting new feature brings out the best of elasticity capabilities in both IBM Workload Deployer and WebSphere Virtual Enterprise. The result is a single management plane that gives you both VM and JVM elasticity for your cloud-based application environments. Best of all, elasticity actions map directly to SLAs for your applications. After all, when it comes to cloud, it's the application that really matters!
I just wanted to point out a great opportunity for anybody considering leveraging IBM Workload Deployer v3 to deploy Database workloads. On June 29th Rav Ahuja, a Senior Product Manager for Data Management at IBM, will be hosting a webcast entitled "Easily Deploying Private Clouds for Database Workloads". He will be joined by Chris Gruber (Product Manager, Database as a Service), Leon Katsnelson (Program Director, IM Cloud Computing Center of Competence), and Sal Vella (Vice President, Database Development and Warehousing) in this panel discussion.
As many of you already know, IBM Workload Deployer v3 comes pre-loaded with DB2 images and patterns that are configured to rapidly provision standardized database servers for any number of purposes. The servers can be deployed in standalone configurations or as part of a complete virtual system including web components with the database components. These servers can also be configured for high availability scenarios. This panel discussion will cover all of these scenarios and more.
You can read more about the webcast in this blog post by Rav Ahuja.
If you want further details about how to build and rapidly deploy databases in a private cloud, be sure to attend this free webinar on June 29th.
We've been talking a lot about IBM Workload Deployer V3 and we will continue to highlight different aspects of the capabilities it provides in the coming weeks. As we've already mentioned - IBM® Workload Deployer V3 is not just another release of the IBM WebSphere CloudBurst Appliance. While it builds on WebSphere CloudBurst's success, and supports and improves upon all of its original capabilities, Workload Deployer provides new application-centric computing capabilities for your private cloud, and brings you higher utilization, improved ease of use, and more rapid application deployment.
In my opinion, declarative deployment models are key to the entire notion of Platform as a Service (PaaS). That is, users should concern themselves with what they want, but not necessarily how to get it. The PaaS system should be able to interpret imperatives from the user and automatically convert that to a running system. In this respect, I think the new virtual application pattern, and more specifically policies, in IBM Workload Deployer takes a giant leap toward a more declarative deployment model.
In IBM Workload Deployer, policies allow you to 'decorate' your virtual application pattern with functional and non-functional requirements. In other words, they provide a vehicle for you to tell the system what qualities of service you expect for your application environment. To put a little context around this discussion, let's examine the policies available in the virtual application pattern for web applications. Specifically, let's look at the four policy types you can attach to Enterprise Application, Web Application, and OSGI Application components in this pattern:
Scaling policy: When it comes to cloud, the first thing many folks think about is autonomic elasticity. Applications should scale up and down based on criteria defined by the user. Well, that is exactly what the scaling policy lets you do. You simply attach this policy to your application component, and then specify properties that define when to scale. First, you choose a scaling trigger from a list that includes application response time, CPU usage, JDBC connection wait time, and JDBC connection pool usage. After choosing your trigger, you decide the minimum and maximum number of application instances for your deployment, and then you choose the minimum number of seconds to wait for an add or remove action. At this point, you can deploy your application and IBM Workload Deployer will monitor the environment, automatically triggering scaling actions as needed.
JVM policy: I would be willing to bet that nearly all of you tune the JVM environment into which you deploy your applications. The JVM policy allows you to take two common tuning actions, setting the JVM heap sizes and passing in JVM arguments, as well as attach a debugger to the Java process (especially useful in development and test phases). You can also use the policy to enable verbose garbage collection (invaluable to understanding heap usage patterns for your application) and select the bit level (from 32 or 64) for your application. Again, all you have to do is attach the policy and specify the properties. IBM Workload Deployer will take care of the required configuration updates.
Routing policy: The routing policy provides a simple way to specify virtual hostnames and allowable protocols (HTTP or HTTPS) for your application. Attach the policy, provide the virtual hostname you want to use, select the desired protocols, and that's it! Remember, once you set the virtual hostname you will need to update your name server to map the hostname to the appropriate IP address.
Log policy: During the development and test phase, it is likely that you will want to enable certain trace strings in the application runtime. The log policy allows you to provide trace strings for your application, and it makes sure that the appropriate configuration updates occur in the deployed environment.
While this is not an exhaustive explanation of each of the policies above, I hope it gives you a basic idea of what they are and how to use them. To me, declarative deployment models are going to be a crucial part of making PaaS successful, so I am really excited about the notion of policies in IBM Workload Deployer. What do you think?
A few weeks ago, I had a conversation with a current WebSphere customer about the potential value they could derive from the use of IBM Workload Deployer. Right away, this customer saw value in the consistency that a patterns-based approach could afford them. It was clear that patterns eliminate the uncertainty that can make its way into even the best-planned deployment processes. Initially though, the customer questioned the value of being able to do fast deployments because, in their words, "We don't deploy WebSphere environments that often." So, we continued our discussion, and then they asked an important question that I encourage all of our users to ask: "Why don't we deploy our WebSphere environments more frequently?"
It is interesting to talk with our WebSphere users that have a long history with our products. Often times, they have been taking a shared approach to WebSphere installations for many, many years. They develop innovative approaches and isolation schemes that allow them to carve up a single WebSphere installation (cell) amongst multiple different application teams. This allows them to avoid having to setup a cell for each application deployment and saves them the associated time. However, having talked to many different users taking this approach, it is not without its challenges.
As was the case in the customer I mention above, users typically made trade-offs when electing for larger, shared cells. As an example, if you have multiple different application teams with different types of applications using a single cell, applying fixes and upgrades to that cell can be a lot more complex. After all, you now have to coordinate plans across a number of different teams and find a window that fits all of their needs. For the same reason, trying incremental function via our feature packs is much more arduous in these types of cells. Additionally, administrative controls become more complex since teams with varying needs all require administrative access. Admittedly, this gets simpler with newer fine-grained security models in WebSphere Application Server v7 and v8, but it still requires organizational discipline and process.
At this point I should be clear that I am not denigrating the shared cell approach. It can work well, and we have many facilities built into the WebSphere Application Server product to support that model. However, if you are using this approach and you find yourself stumbling too much for your own liking, then I would strongly suggest that you explore the patterns-based approach of IBM Workload Deployer. By deploying patterns that represent your WebSphere cells using IBM Workload Deployer, you can quickly and consistently setup multiple WebSphere Application Server cells to support the varying needs of your application teams. You will still avoid spending an inordinate amount of time installing and configuring cells as that is an automated part of pattern deployment, and your application teams will still get the resources they need. Further, this can liberate your application teams in terms of how they apply maintenance, install upgrades, and absorb new function in the form of feature packs.
I am not suggesting a complete pendulum swing in your approach to how you manage multiple application environments. There is definitely a happy medium in terms of how many cells you end up with. After all, you do not want to trade in one set of problems for the problem of managing way too many different cells. However, I do think that decomposing monolithic, multi-purpose cells into smaller, more purposeful cells can be beneficial. In the course of thinking about this different approach, you may come to the same conclusion that the customer I mention above did. IBM Workload Deployer's rapid deployment capabilities are indeed valuable if you take a slightly different view of current processes.