IBM Impact 2012 was just last week with a theme of "Change the Game" ... and I'm still reveling in all of the excitement and energy that goes into conferences such as this. I was fortunate to get a last minute spot to attend the conference and help out at the Solution Center where I had the chance to speak to a lot of customers and other IBMers interested in cloud computing. Among the many things that stood out - there is certainly a lot of interest in cloud computing and patterns of expertise - it also seems that folks are ready to get some first hand experience with these patterns. There's plenty of opportunity for that!
I write a lot about WebSphere CloudBurst script packages. Typically, I write about what they are, how to create them, and even provide some samples from time to time. I find that most of the time I'm either writing or talking about script packages from the standpoint that they allow you to automate the delivery of customizations to environments you deploy with WebSphere CloudBurst. More specifically, I usually explain how you can include these script packages in your patterns to ensure that your custom scripts execute as part of every pattern deployment. The truth is, that is not the whole story. In fact, it's only 1/3 of the story.
In WebSphere CloudBurst, when you define a script package you also define its execution mode. The execution mode can be one of three values, and it indicates the invocation time for the script. The default value is at virtual system creation, and that tells WebSphere CloudBurst to automatically invoke the script as part of the deployment process. This seems to be the most commonly used execution mode, and in the original version of WebSphere CloudBurst it was the only available mode (which probably attributes to why I only usually tell 1/3 of the story here). As you may expect, there is a wide range of usage scenarios for this class of script packages including installing applications, activating monitoring agents, registering cells with an externally managed DNS server, and much more.
If you are like me (and many humans), you enjoy and actually expect symmetry. In that regard, it probably comes as no surprise that there is a script package execution mode called at virtual system deletion. As the name indicates, this class of script packages executes as an automatic part of the virtual system deletion process. When a user tells WebSphere CloudBurst to remove a virtual system, before it shuts down the machines in the system, it will run each script package marked to execute at virtual system deletion. Typical use cases for these scripts include removing information about the cell from externally managed DNS servers, freeing up connections with external systems, and other external 'clean up' activities.
So this leaves the final execution mode for script packages, the when I initiate it mode to be precise. This class of script packages executes when explicitly triggered by a user. In the virtual machine detail section for a deployed virtual system, you can see a list of user-initiated script packages for a given machine. There is a start button by each of the user-initiated script packages that allows you to invoke the script when, and as many times as you need to. While these script packages have many different use cases, the most common use case is to deploy application updates. Users build these application update scripts, attach them to a pattern, and invoke them whenever they want to deliver an updated application into their already deployed environment.
WebSphere CloudBurst script packages are one of the main vehicles for delivering your customizations to your cloud environments. The three execution modes mentioned above allow you to determine when the right time to deliver those customizations is.
The 1.1.1 version of WebSphere CloudBurst is now available on the IBM support site, and you can read a bit about it here. I have not blogged about each and every point release of WebSphere CloudBurst here, but this particular one is significant. The 1.1.1 release adds support for IBM's z/VM hypervisor platform as a deployment target.
Quite simply this means that you can now create virtualized WebSphere application environments on the z/VM platform using WebSphere CloudBurst. This starts by first defining target hypervisors, instances of Linux Master Systems, that WebSphere CloudBurst can communicate with to setup virtual machines on z/VM. Coupled with support for the z/VM infrastructure components, there is a new version of the WebSphere Application Server Hypervisor Edition virtual image. While this new image contains the same basic components as the previous hypervisor edition images, it is packaged for the z/VM platform, and it includes a zLinux operating system.
Once you upgrade to WebSphere CloudBurst 1.1.1, you can define your z/VM infrastructure and use the new WebSphere Application Server Hypervisor Edition image to build custom patterns targeted for the z/VM platform. I put together a demonstration that provides a very brief overview of these new capabilities. As always, please reach out if you have any questions about this or any other WebSphere CloudBurst topic.
It does not seem like it has been a year, but here we are again. It is time for IBM Impact 2012, and like each year, this one promises to be a little better than its predecessors. As I type this post, I am 36,000 feet above either Texas or New Mexico on my way to the neon desert for a completely packed week. I can't wait to arrive!
An obvious summary of IBM Impact would be to say that it is a technology conference. That does not quite do it justice though. The event is packed full with stories of business transformation, emerging business paradigms and the technologies that support them, new product announcements and much more. That said, in my mind IBM Impact is first and foremost a premier technical education conference. The week is stuffed with technical session after technical session on a wide range of topics.
With that in mind, I thought I would share a few of the sessions I have highlighted on my calendar. To be honest, I had a hard time setting up my calendar for the week. In some cases I ended up booking three sessions in one time slot. There are simply too many good sessions to choose from, so this list is nowhere near exhaustive, and it is certainly not my full calendar!
1219 Overview of the IBM Mobile Foundation :: Monday 10:45 AM - 12:00 PM :: Palazzo N - Venetian
Summary: This session will provide an overview of the new IBM Mobile Foundation: a new middleware offering from IBM that will enable customers to build and deliver innovative mobile applications, centrally govern and manage their mobile infrastructure, and integrate with existing enterprise data and services. Attendees will leave with an understanding of what the platform is and how it can help them effectively and efficiency take advantage of mobile for their enterprise.
2138 Elastic Caching - Foundational Technology for Your Solutions and Offerings :: Monday 2:00 PM - 3:15 PM :: Palazzo J - Venetian
Summary: This session will provide an overview of elastic caching, explain IBM's offerings and technology and will share a set of usage scenarios that will demonstrate why this technology is so hot -- and why it can dramatically benefit our partners' offerings, solutions and ROI.
1371 Introducing IBM WebSphere Application Server v8.Next - Enhanced ND: A Huge Step Forward! :: Tuesday 10:45 AM - 12:00 PM :: Palazzo I - Venetian
Summary: Setting the bar higher for app server resiliency and robustness, IBM WebSphere Application Server Network Deployment v8.Next sets itself even further apart from the industry. Now included with WAS ND is virtualization, improved availability and health monitoring, Java batch processing and more. This session covers the details.
2484 Cloud, Virtualization and Application Pattern Trends and Directions :: Tuesday 1:30 PM - 2:45 PM :: Palazzo E - Venetian
Summary: Cloud and virtualization is being pared with a new best practices-based approach to application development, deployment and automation of custom and independent software vendor applications across a range of deployment environments. Whether you're targeting existing hardware and software stacks, new private cloud infrastructure or public cloud resources, or all three, a pattern-based approach to applications can deliver unmatched portability and time to market. Cloud computing is helping to deliver a level of automation and self service needed in todays dynamic business landscape. Learn how these technologies are unfolding, and what your company can do to get started today to drive speed, efficiency and lower total cost of ownership across your IT investments.
1520 Building Custom Content for Expert Integrated Systems :: Wednesday 10:45 AM - 12:00 PM :: Palazzo G - Venetian
Summary: This session will cover all the different ways that new functionality can be developed for use with IBM Workload Deployer and IBM PureApplication System. This will include the ICON Image Construction tool, Capture and Extend, and also the IWD PDK.
1563 Positioning Expert Integrated Systems with Its Competitors :: Wednesday 3:15 PM - 4:30 PM :: Palazzo F - Venetian
Summary: (Plug Alert! -- I will be co-presenting this session) IBM PureApplication System provides a virtualized platform of cost effective, next generation hardware with an optimized capabilities that automates workload lifecycle management, from deployment to quiescence. How is it different from other solutions that promise the same benefits? Join us as we examine the unique capabilities of IBM PureApplication System. Learn about the value of IBM PureApplication System for your business, and why it is truly heads and shoulders above the competition! Explore in detail why IBM PureApplication System can better deliver on these capabilities than other alternatives such as Oracle's Exalogic. Comparisons will be based on quantitative metrics using results from actual experiences with both products.
2150 Building a Private Cloud Using IBM Technology and Fit-for-Purpose Methodology :: Wednesday 4:45 PM - 6:00 PM :: Marcello 4401A - Venetian
Summary: In this session, we will examine a practical approach for organizations to optimize their computing environment by using IBM WebSphere technology. We explore the use of IBM's fit for purpose methodology for optimal workload placement, how the use of IBM's Workload Deployer, IBM Datapower, the IBM Rational Automation Framework for WebSphere and WebSphere Virtual Enterprise can be used in the development of a private cloud that maximizes your total computing environment.
2390 WAS vs. WebLogic, JBoss and Tomcat: An IBM Perspective :: Thursday 8:45 AM - 10:00 AM :: Lido 3105 - Venetian
Summary: Are you considering an Oracle WebLogic or an open source application server like Tomcat or JBoss? In this session we will discuss key factors to consider when making a decision on which application server to use, such as cost of licenses and support, performance, availability and usability lab tests, administrative and development tools, and real world customer experiences. We will discuss factors that contribute to TCO such as development and operating costs, and application performance and reliability. We will discuss how new capabilities of WAS v8.Next enhance its competitive position. Session will be presented by Roman Kharkovski, who has been a technical lead on the WW WebSphere Competitive Team since 1999 and Stuart Smith, who is lead consultant with Web Age Solutions and worked with all major application servers and Java since 1998.
Summary: eXtremeMemory allows you to store objects in native memory instead of on the Java heap. By moving objects off the Java heap, you can avoid garbage collection pauses, leading to more consistent performance and predictable response times.
These are just a few of the interesting sessions I have highlighted on my calendar. I am going to sit in on many more, and I will be writing a summary of the event soon enough. For those of you heading to IBM Impact, safe travels and I hope to see you there!
One of the things that often comes up at some point in IBM Workload Deployer conversations is the notion of self-service access. Specifically, users want to know what the appliance provides that enables them to allow various teams in their organization to directly deploy the middleware environments they need. In other words, they want to use IBM Workload Deployer to tear down the traditional barriers that exist between those that request the environment and those that fulfill said request. Now, as we begin to elaborate on this notion, it becomes quickly apparent that in order to effectively enable self-service, IBM Workload Deployer must deliver a few things.
First, IBM Workload Deployer must provide the means to define users with various levels of access. Second, IBM Workload Deployer must provide the means to define resource access at a fine-grained level to different users and groups of users. Check and check. The appliance has been doing this since the beginning of WebSphere CloudBurst. Without those two things, the conversation of self-service access would end pretty quickly. However, there is a final capability that is equally important: IBM Workload Deployer must deliver a means to limit resource consumption at a fine-grained level.
In IBM Workload Deployer there are a couple of ways to achieve this. First, you could define multiple cloud groups and allow access to those groups in a way that maps directly to resource entitlements. While that may work in some situations, others call for even more granularity. You may want to allow multiple different users or groups to access a cloud group, but you may want to allow different consumption limits for each of these groups. In this situation, you can take advantage of environment profiles and a new option when defining users of IBM Workload Deployer.
Consider the case that you have a group of developers and you want to limit their consumption of memory in the cloud. First, you start by defining your development users and for each you select Environment Profile Only as the value for the Deployment Options field.
By selecting the above value for the deployment options of a user, you restrict that user to only deploying via an environment profile as opposed to general cloud group deployments. After defining all of your development users, you may choose to organize them into a user group for easier management. At that point, you can define environment profiles and determine which ones your developers should have access to using the Access granted to field of the profile.
Within the environment profile, you can define resource consumption limits for compute resource and software licenses. For instance, you can define a limit on the amount of virtual memory consumed by all deployments using the profile. It is important to note that the limit is cumulative for ALL deployments that use the profile.
Now that all of the controls are in place, consider the deployment process for one of your development users. They pick a virtual system pattern, click the deploy icon and begin to configure the pattern for deployment. In the Choose Environment section of the deployment dialog, your development user will only be able to select the Choose profile option for deployment. Further, they will only be able to deploy using the environment profiles to which they have access.
After the deployment completes, a look at the Environment limits section in the profile shows the current usage totals.
Now suppose another development user, or even the same one, comes along and attempts to deploy another virtual system pattern even though the profile limits have already been reached. The user can initiate the deployment, but they will get a near immediate failure owing to the fact that they would exceed consumption limits if the deployment were allowed to proceed.
The same kind of enforcement occurs regardless of the resource limit type. You can use this approach to limit the consumption of CPU, virtual memory, storage, or software licenses among the various different users or groups of users you define in IBM Workload Deployer. If you combine fine-grained resource consumption limits with varying permissions and fine-grained access, I think you are on the road to truly enabling self-service in the enterprise.
The reason I suggest the application proxy approach is twofold. First, it affords you the ability of having custom interactions with the REST API. For instance, you may insert logic into the server-side proxy code that returns only a subset of the JSON data contained in the response from the appliance. Alternatively, in an effort to reduce the chattiness on your client-side, you may join JSON data from multiple different REST requests to the appliance to fulfill a single client request. You may even decide to represent the data in an all together different format than JSON. All of these options and many more are available to you if you implement an application-based proxy to the REST API.
The second reason I suggest the application approach is that it is easier, and seemingly safer, to not deal with user passwords on the client-side. If you setup your application proxy, you can configure it to retrieve the appropriate password from a secure location (like an encoded file) based on information passed along in the request. This means the password information is only present in the request (in encoded form of course) from the application proxy to the WebSphere CloudBurst Appliance.
The good news about the application-based proxy approach is that it is simple to put in place. I composed one using the open source Apache Wink project. The Apache Wink project is an open source implementation of the JAX-RS specification (and then some), and it enables you to develop POJOs that are in turn exposed in a RESTful manner. In my case, I had a single resource POJO:
The Apache Wink runtime routes any HTTP GET request whose path is like /resources/* to the getResources method in the WCAResource class. This method passes along information taken from the query string (the host name of the target WebSphere CloudBurst Appliance and the requesting WebSphere CloudBurst username), as well as the HTTP path information and sends it on to the getResource method declared as follows:
The getResource method above uses the WebSphere CloudBurst host name and the request path to construct the URL for the corresponding WebSphere CloudBurst REST API call. Next, it constructs an Apache Wink Resource object and sends the REST request along to the WebSphere CloudBurst Appliance. How do we authenticate this request? We use the WebSphere CloudBurst username (sent as a query string parameter) to retrieve the appropriate encoded password information. Once we have that, we construct the necessary header for basic authorization over SSL.
The application-based proxy shown here is simply a pass-through. It does not manipulate the data returned from the WebSphere CloudBurst REST API, nor does it map a single client-side call to multiple REST requests. However, it would be simple enough to extend it to do any of those things. If you have any questions about the code here, please let me know. I'd be happy to share more of the code, or talk about how and where to extend it.
To continue with the series of blog posts regarding WebSphere CloudBurst FAQs, I want to take a look at one aspect of the deployment process.
When you leverage WebSphere CloudBurst to push patterns (complete WebSphere Application Server configurations) into a private cloud, the appliance provides an advanced placement algorithm to determine exactly where the resulting WebSphere virtual systems will reside. It attempts to match the needs of the pattern to the correct set of hypervisors that have been defined. WebSphere CloudBurst considers things like storage, CPU, memory, and high availability characteristics when placing the pattern, and this is all done by the appliance without you having to intervene at all.
This is certainly nice in that it absolves you from having to make such placement decisions. Having said this though, you may be thinking of a question that comes up quite often:
If WebSphere CloudBurst controls the placement of the pattern, how can I make sure that certain deployments end up on certain servers (hypervisors)?
Considering what I just told you above, it may not seem that it's possible to control what machines end up hosting your virtual system since the appliance takes care of that placement for you. However, the organized use of WebSphere CloudBurst cloud groups allows you to take advantage of the intelligent placement provided by the appliance while retaining a level of control over which machines end up hosting particular deployments.
In WebSphere CloudBurst all patterns are deployed to cloud groups. Cloud groups are a collection of hypervisors that have been defined within the appliance. The basic deployment mapping is depicted in the image below:
As seen above, you can create a cloud group for any purpose (dev, test, QA, production, etc.), including any hypervisors that you desire as long as a given hypervisor only belongs to a single cloud group. When you are ready to deploy a pattern, you simply select the cloud group you want to deploy to:
By selecting a cloud group for deployment, you are implicitly selecting the physical machines that will host your deployment. The cloud group could consist of anywhere from one to N hypervisors, so you are afforded the ability to restrict the location of your virtual systems as necessary.
I hope this helped explain a little bit about cloud groups in WebSphere CloudBurst. If you're looking for more information about WebSphere CloudBurst cloud groups, I'd also suggest you watch this video on our YouTube channel.
Among the major features of the new virtual application pattern in IBM Workload Deployer is the notion of elasticity. That is, as your application needs more resources, it gets them. When your application can meet its SLAs with fewer resources, the environment shrinks. With this kind of pattern, you enable elasticity by specifying a policy and defining the scaling trigger (i.e. CPU usage, application response times, database response times, etc.). What may have been a bit lost in some of these new announcements regarding IBM Workload Deployer is the fact that you can now leverage this core feature of cloud, elasticity, in your virtual system patterns.
If you have read this blog in the past, you probably already know that the Intelligent Management Pack is an option for virtual system patterns built using WebSphere Application Server Hypervisor Edition. When you enable the Intelligent Management Pack option, you are essentially building and deploying WebSphere Virtual Enterprise (WVE) environments. For those of you not familiar with WVE, the best way to describe it is that it provides you with application and application infrastructure virtualization capabilities. Of its many capabilities, one most germane to our discussion today is the ability for users to attach SLAs to applications and then have WVE automatically prioritize requests and manage resources in order to meet those SLAs. Inherent in this capability is the ability to dynamically start and stop application server processes (JVMs) as required. In other words, WVE provides JVM elasticity.
The fact that WVE provides JVM elasticity is nothing new. Further, IBM Workload Deployer started providing virtual machine (VM) elasticity in previous versions (when it was WebSphere CloudBurst). With this feature, you could add or remove VMs to an already deployed virtual system using dynamic virtual machine operations provided by the appliance. The catch was that the VM elasticity was a manual action and you could not link this elasticity to the same SLAs tied to your applications. Well, thanks to a new feature in WebSphere Virtual Enterprise and easy integration provided by the Intelligent Management Pack, this is no longer the case.
Starting in IBM Workload Deployer 3.0, you can take advantage of a new WVE feature called Elasticity Mode when using the Intelligent Management Pack. Elasticity mode is not unique to IBM Workload Deployer, but a concept new to the base WVE product. It allows one to define actions for how WVE should grow and shrink the set of nodes used by application server resources. Like the basic JVM elasticity capability in WVE, these node elasticity actions trigger based on SLAs tied to your applications. Consider the case that you are using elasticity mode and your application is not currently meetings its SLA. If WVE does not think it can start any more application server instances on the current set of nodes, it will grow the set of nodes per your elasticity configuration. Conversely, if WVE detects that it can meet SLAs with fewer nodes, it will shrink the resources per your elasticity configuration.
In IBM Workload Deployer, using elasticity mode becomes even easier. All you need to do is use the Intelligent Management Pack and enable the elasticity mode option in your virtual system patterns. When you do this, you get automatic integration between IBM Workload Deployer and the deployed WVE environment. What does that mean? It means that if WVE detects it needs more nodes, it will automatically call back into IBM Workload Deployer and request that the appliance provision a new VM that will serve as a node for application server processes. It also means that if WVE detects it could meet SLAs with fewer resources, it will call into IBM Workload Deployer and ask it to remove a node. All of this happens without any user scripting. All you have to do is enable this option in your patterns and configure SLAs appropriate for your applications.
To me, this exciting new feature brings out the best of elasticity capabilities in both IBM Workload Deployer and WebSphere Virtual Enterprise. The result is a single management plane that gives you both VM and JVM elasticity for your cloud-based application environments. Best of all, elasticity actions map directly to SLAs for your applications. After all, when it comes to cloud, it's the application that really matters!
Customers are always impressed when they learn about the simplicity, resiliency, and rapid time to value they can received from virtual applications. However, they are usually a little mystified at how virtual applications really work. After all - they have become quite accustomed to doing things the "traditional way" where they control every aspect of their applications manually. Virtual Applications represent an entirely new way of thinking. Sure, the benefits are enormous but can you really trust them? How is it doing all of this anyway?
What seems like "magic" is really a sophisticated and coordinated set of activities driven and coordinated by IBM Workload Deployer while leveraging the expertise built into the pattern type. Yes, you can trust it because experts have worked to build the system and created to it react and respond much faster than you can. When moving away from manual processes to automated processes it is always nice to get a sense of what is really happening. I think it is just human nature. We can't really place our trust in something until we have first hand experience or understand what it is really doing ... I guess it is the critic inside each one of us. Even after you've experienced the value it is still reassuring to see and understand the "how".
It is the "how does it do that?" type of question that I attempted to answer for virtual applications in a blog post I wrote on the Expert Integrated Systems blog recently. It attempts to pull the curtain aside and describe what is actually happening to support a virtual application pattern. As with my previous post - this was written for IBM PureApplication Systems but the concepts are 100% applicable to IBM Workload Deployer. I think you will find it interesting ... Continue reading ...
Script packages are an integral part of virtual system patterns in IBM Workload Deployer. By attaching script packages to your patterns, you provide customizations particular to your unique cloud-based middleware environments. Customizations provided by script packages might include installing applications, creating application resources, integrating with external enterprise systems, and much more. The bottom line is, if you are creating virtual system patterns, you will almost certainly be creating script packages.
Largely, the act of creating a script package is independent of IBM Workload Deployer. The appliance does not dictate a particular scripting language, so all you need to do is make sure you can invoke your logic in the operating system environment. Your script package may be a wsadmin script, shell script, Java program, Perl script, and on and on. After you create the actual contents of your script package, you will then load that asset into the IBM Workload Deployer catalog.
Once loaded into the catalog, you define several attributes of your script package, including the executable command, command arguments, variables, execution time, and more. The process for defining these attributes is trivial using the intuitive UI in IBM Workload Deployer, but I wanted to take a little time to remind you of a technique I recommend to all users defining script packages. You can actually package a JSON file within the script package that defines all of the script's attributes. The format of the file is simple, and I am including an example below:
The example above is one taken from a script package in our samples gallery, and it shows the basics of which you need to be aware. Notice that in the JSON file, you can provide a name, description, unzip location, executable command, command arguments, variables, and more. You only need to ensure that the name of this JSON file is cbscript.json and that you include it at the root of the script package archive. Once you have done that, you load the script package archive into the catalog, refresh the script package details, and voila -- all the attribute definitions appear!
You may ask why I recommend this since it could seem like an unnecessary step. My answer to that is that you have to define these attributes anyway, so you might as well capture it once in the file. Once you capture it once in the file, you can ensure that if the same script needs to be reloaded, or if you need to move it to another appliance, its definition will be exactly the same (and presumably correct). I use this approach for all of my work, and for all of the samples I contribute to our gallery, and it really saves me a lot of misplaced effort that can result from typos. If you are out there creating script packages, try adopting this approach. I'm pretty sure you will be happy you did!
A while back I co-authored an article along with Chris Ahl from Tivoli and Ken Klingensmith from WebSphere Technical Sales about the customization of virtual images in WebSphere CloudBurst. In the article we approached image customization as a means to enable IBM Tivoli Monitoring for the operating system within virtual machines dispensed by WebSphere CloudBurst. Today I posted a short demonstration that discusses and shows this particular integration scenario. If you are interested, but haven't had time to read the article, you may want to watch the video first as it should give you a good overview of the process and results.
Talk of Tivoli reminds me that IBM Pulse 2010 is just around the corner. I'll be going to discuss WebSphere CloudBurst and how it can be paired with software from IBM Tivoli for high-value integration scenarios. In the session I'll be talking about the Tivoli Monitoring integration as well as other key points such as our integration with Tivoli Service Automation Manager, IBM CloudBurst, and more. The best part about the session is that I will be co-presenting alongside a WebSphere CloudBurst customer that will dole out practical advice for using WebSphere CloudBurst within the enterprise. Join us on Tuesday February 23rd from 3:30 - 4:30 in Conference Center 306.
Remember, any time you have questions about WebSphere CloudBurst please pass them along. You can leave comments on this blog, or you can reach me at my new Twitter location @damrhein.
I was very encouraged by the consistently positive response we got at IMPACT for our WebSphere CloudBurst and Rational Automation Framework for WebSphere (RAFW) integration. I believe there were many reasons for this response: accelerated time to value, decreased investment needs for activities that are not core to your business, lowered barrier of entry for provisioning and configuring WebSphere cells, and much more. While those are certainly all very real and valuable benefits, I also believe that quite a bit of interest in this integrated solution comes from the fact that it is applicable to a number of needs common to you, our WebSphere users.
With that in mind, let's look at some (not all) of the scenarios where WebSphere CloudBurst and RAFW integration can help you:
Create WebSphere CloudBurst patterns that include configuration without scripting: Users love our WebSphere CloudBurst patterns. They really see the value in codifying both the topology and configuration of their application infrastructure. However, some users do not have existing WebSphere configuration scripts and do not have the time and/or resource to invest in creating these scripts. They are looking for a solution that provides not only the provisioning of WebSphere environments but also the configuration of said environments (configuration beyond what our IBM Hypervisor Edition images encapsulate, specifically application-oriented configuration). RAFW provides this capability in the form of 500+ out-of-the-box configuration actions for WebSphere environments. This includes actions to create JDBC resources, create JMS queues, deploy applications, configure application containers, and much, much more. You can create WebSphere CloudBurst patterns that contain a special script package, which points back to a RAFW project containing a set of configuration actions. During deployment, WebSphere CloudBurst will provision your WebSphere environment and then cause the invocation of the specified RAFW project, which in turn runs a set of configuration actions against the provisioned environment. This means you can set up full-blown, ready-to-go application environments with absolutely no user-supplied scripting. In fact, I took this approach to setup a J2EE performance benchmark application, DayTrader 2.0, running on WebSphere Application Server. Those of you familiar with the application know this is not a trivial environment to stand up. Yet, I did it without having to personally write a single line of configuration scripting, and it was all ready to go in around thirty minutes.
Creating WebSphere CloudBurst patterns from existing environments: This comes up all the time. I go through a standard introduction to WebSphere CloudBurst, users see the value, love the patterns-based approach, and immediately want to know how they get their existing environments into the form of a pattern. RAFW, along with the special WebSphere CloudBurst script package, can make this a straightforward and hardened process. You use a capability in RAFW to import the configuration of an existing cell, thereby creating a RAFW environment for that configuration. You then create a WebSphere CloudBurst pattern with a topology congruent to your existing environment, attach the special script package I mentioned earlier, and you are done with the import! When you deploy this pattern, you simply specify the RAFW environment that you created earlier (the one that holds the configuration data for your existing environment) and a RAFW project that will apply the configuration data in that RAFW environment to the WebSphere environment provisioned by WebSphere CloudBurst. The creation of the WebSphere environment, as well as its configuration, happens in a completely automated fashion.
Configure, capture, reuse: There are many situations that may require you to make manual changes to a WebSphere cell after it has been deployed. For example, during performance testing for your application, you might discover that you need to tweak the number of available threads in the web container. As another example, for the first setup of a given application environment, you may want to quickly deploy the cell using WebSphere CloudBurst and then manually install and configure your applications to make sure everything is just right. In either case, it is likely that you want to capture the updated configuration and make sure that any future deployments use those updates. Again, WebSphere CloudBurst and RAFW makes this simple. First, you build a pattern that encapsulates your WebSphere topology (the types and quantity of nodes you want) and attach the special script package mentioned above. For the first deployment, you simply specify the name of the new RAFW environment you want to create. Once the system is up, you log into the WebSphere administration console, make your necessary customizations, and then you use RAFW to import that updated configuration thus updating the initially created RAFW environment. For subsequent deployments, you simply deploy the same pattern, specifying the same RAFW environment as well as a RAFW project, which RAFW automatically created for you during the first deployment. This project applies the configuration (the one you manually established and imported into RAFW) to the WebSphere environment setup by WebSphere CloudBurst.
Configure WebSphere environments across virtual and physical settings: It seems that in many cases our users manage the same WebSphere environment across both virtual and physical settings. For example, they may provision the application environment using WebSphere CloudBurst for everything from development to pre-production, and then for production provision that same environment to a set of physical servers. At least, they try to provision the same environment. In reality, it is tough to reproduce the exact same configuration once you break from the WebSphere CloudBurst patterns-based approach. However, if you stored the configuration of your WebSphere cell as a RAFW environment, you could apply that configuration data to a WebSphere cell regardless of whether it existed in the physical or virtual world. Once you move to physical, you do lose out on the fast provisioning, WebSphere intelligence, cloud management capabilities, and automated integration with RAFW that you get when using WebSphere CloudBurst, but if it is in your process to move to physical hardware at some point, reusing the same RAFW environment certainly eases the migration task.
I hope this sheds some light on some of the common issues WebSphere CloudBurst and the Rational Automation Framework for WebSphere can combine to solve really well. This is by no means an exhaustive list, but really meant to point out the broad application of the solution. If you want to see how it works, check out this video.
May is almost here and that means that IBM IMPACT is right around the corner. Just like years past, IMPACT 2010 will be a great chance to get valuable education and insight into IBM WebSphere software and software from across the IBM software family. If you want to hear how IBM software is leading the march toward a smarter planet, register now.
IMPACT 2010 will be a great chance to hear the WebSphere cloud computing story. There will be multiple sessions on the WebSphere CloudBurst Appliance. These include customer-led sessions, internal adoption stories, overviews, and much more. I'll be there running a hands-on lab and delivering a session that discusses integration between WebSphere CloudBurst and IBM Rational tools. Of course, there is more to WebSphere and cloud computing than WebSphere CloudBurst. We have several other sessions that will detail all of IBM WebSphere's work in the cloud.
If you are interested, I put together a short video discussing some of the sessions on tap for WebSphere and cloud computing at IMPACT 2010. I'd also encourage you to check out the social media site for IBM IMPACT 2010. On that site, you will find tweets, videos, and blogs about the conference. Don't forget to sign up, and I hope to see you in Las Vegas!
-- Dustin Amrhein
If you are reading this blog then I am pretty sure that you are interested in the agility that can be achieved by rapidly provisioning middleware systems and standing up virtual applications in a private cloud environment. However there are other aspects of agility that you should also consider. One such aspect is the ability to build applications that can be easily maintained, updated, and extended. This is where OSGi technology comes into the picture.
If you have been working with the IBM Workload Deployer (or watching some IBM Workload Deployer demos) you may have noticed a category of components in the virtual application builder called OSGi Components.
Maybe you already know all about OSGi applications and the value they bring to an enterprise. Or, perhaps you noticed this and decided that you would search for some more information on this odd acronym and just what an OSGi application is all about.
In a nutshell OSGi technology is a way to define dynamic modules for Java. It provides a standard way to encapsulate components (called bundles) with metadata that define versioned package dependencies, service dependencies, packages exported, services exported, etc... basically everything you need to know about this bundle so that it can be connected up with other bundles to support a particular solution. These bundles can then be grouped together into applications and dynamically wired to fulfill necessary dependencies at runtime. The OSGi framework provides all of the necessary capability to manage the dependencies and resolve any problems.
Those who leverage OSGi technology benefit from improved time-to-market and reduced development costs. The loose coupling provided by the OSGi framework reduces maintenance costs and facilitates the dynamic delivery of components in a running system. Of course there's a lot more to it than just that ... involving portability across different environments, achieving the appropriate level of isolation or sharing within an environment, and integrating with the many different technologies and patterns already available today. I don't think I know enough about OSGi to do it justice here. But fortunately for me (and you) there are several experts who can make it all clear.
One such expert is Graham Charters and there is a great opportunity to hear him introduce this topic and also participate in a dialogue about the concepts and what they mean for your business. Graham will be leading a Global WebSphere Community Lab Chat on Wednesday of this week (July 20th) entitled: How can OSGi make your enterprise more agile. Graham is the IBM technical lead in the OSGi Alliance Enterprise Expert Group and an active participant in the open source community implementing many of these standards. So register now for this free session and learn how OSGi can make your enterprise even more agile.
I’m going to take a different approach this week in the blog. Instead of me telling you about some of the features or uses of WebSphere CloudBurst, I thought I would catch up with someone using the product everyday, WebSphere Test Architect Robbie Minshall. Robbie is responsible for a team of testers that harness a lab of over 2,000 physical machines to put our WebSphere Application Server product through some pretty rigorous testing. Toward the beginning of this year Robbie’s team started to leverage the WebSphere CloudBurst Appliance in order to create the WebSphere Application Server environments needed for their testing.
Robbie, can you tell us a little bit about what the WebSphere Application Server test efforts entail?
In WebSphere Application Server development and test we have two primary scenarios. The first is making sure that developers have rapid access to code, test cases and server topologies so that they can write code, test cases and then execute test scenarios on meaningful topologies. The second scenario is an automated daily regression where in response to a build, we provision a massive amount of WebSphere Application Server topologies and execute our automated regression tests.
Previously we have supported these scenarios through the deployment of the Tivoli Provisioning Manager for operating system provisioning, some applications for checking out environments, and then a lot of automation scripts for the silent install and configuration of WebSphere Application Server cells.
Given those scenarios and the existing solution, what are your motivations for setting up a private cloud using WebSphere CloudBurst Appliance?
We are supporting these scenarios through a pretty complicated combination of technologies. These include silent WAS install scripts, wsadmin configuration scripts, a custom hardware leasing application and the utilization of Tivoli Provisioning Manager for OS Provisioning. This solution is working very well for us though as always we are looking for areas to improve, opportunities to simplify and to reduce our dependency on investment in our custom automation scripts. Mainly, there were 3 areas where we wanted to improve our framework: Availability, Utilization and Management. This is why we started looking to the WebSphere CloudBurst Appliance.
Can you expand a bit on what you are looking for in those three areas?
The first focus area we have is availability of environments. We really wanted to lower the entry requirement for the skills and education necessary to get a development or test environment. Setting up these environments has just been too hard, too time consuming, and too error prone. Using WebSphere CloudBurst we can provide an easy push button solution for developers to get on-demand access to the topologies they need.
The second area we are looking for significant improvements on is hardware utilization. Our budgets are tight and in our native automation pools we are only using between 6-12% of the available physical resources. In order to improve this we were looking at leveraging virtualization. WebSphere CloudBurst offers the classic benefit of virtualization with the nice additions of optimized WebSphere Application Server placement and really good topology and pattern management. In our initial experiments we were able to push the hardware utilization up to 90% of physical capacity and consistently were leveraging around 70% of our physical capacity.
Finally we are looking to improve and simplify our management of physical resources and automation. We work in a lot of small agile teams and organizational priorities change from iteration to iteration. Not only does WebSphere CloudBurst allow us to maintain a catalog of topologies or patterns for releases but it also allows us to adjust physical resource allocation to teams through the use of sub clouds or cloud groups.
Basically we felt that WebSphere CloudBurst would improve the availability of application environments, enhance automation, and improve hardware utilization all with very low physical and administrative costs.
What were some of the challenges involved with getting a cloud up and running in your test department?
One of our challenges seems like it would be common to many scenarios, especially in today’s world. Our budget for new hardware to build out our cloud infrastructure was initially very limited. Most cloud infrastructure designs depict very ideal hardware scenarios including SANs, large multicore machines, and private and public networks within a dedicated lab. Quite frankly we did not have the budget to create this from scratch. It was important for us to demonstrate value and data to warrant future investment in dedicated infrastructure. After some performance comparisons we were very happily surprised to see that we could leverage our existing mixed hardware within a distributed cloud. The performance of application environments dispensed by WebSphere CloudBurst on many small existing boxes in comparison to large multicore machines with a SAN was very comparable. This allows us to leverage existing hardware, with minimal investment all the while demonstrating the value and efficiencies of cloud computing. That data in turn has allowed us to obtain new dedicated hardware to iteratively build up a larger lab specifically for use with WebSphere CloudBurst.
Specifically with WebSphere CloudBurst, are there any tips/hints you would offer users getting started with the appliance?
Sure. First, we quickly realized as we added hypervisors to our WebSphere CloudBurst setup it was critical to have someone with network knowledge on hand. This is because the hypervisors came from various sections of our lab, and we really needed people with knowledge of how the network operated in those different sections. Once we had the right people we were able to setup WebSphere CloudBurst and deploy patterns within an hour and a half.
Moving forward we continued to have challenges as we dynamically moved systems between our native hardware pool and our cloud. Occasionally the WebSphere CloudBurst administrator would move a system into the cloud but incorrectly configure the network or storage information. This lead to some misconfigured hypervisors polluting our cloud. We overcame this, quite simply and satisfactorily I may add, by creating some simple WebSphere CloudBurst CLI scripts which add the hypervisors, test them individually, by carrying out a small deployment to that hypervisor, and then move the correctly configured hypervisors into the cloud after verifying success. Misconfigured hypervisors go into a pool for problem determination. This has allowed us to maintain a clean cloud, and we are able to dynamically move our hardware in and out of the cloud to meet our business objectives.
We also use the WebSphere CloudBurst CLI to prime the cloud so to speak. Before using a given hypervisor in our cloud, we execute scripts that ensure each unique virtual image in our catalog has been deployed to each of our hypervisors at least once. When the image is first deployed to a hypervisor, a cache is created on the hypervisor side of the connection, thus meaning subsequent deployments do not require the entire image to be transferred over the wire. This gives us consistent and fast deployment times once we are using a hypervisor in our cloud.
I would assume that like many applications deployed on WebSphere Application Server, your team’s applications have several external dependencies. Some of these dependencies won’t necessarily be in the cloud, so how did you handle this?
You’re right about the external dependencies. Our applications and test cases run on the WebSphere Application Server but are dependent upon many external resources such as databases, LDAP servers, external web services etc. WebSphere CloudBurst allows us to deploy WAS topologies in a very dynamic and configurable way but the 1.0.1 version does not allow us to deploy these external resources in the same manner. This was overcome by using script packages in our patterns. These script packages allow us to associate our test applications with various patterns we have defined. The script package definition also allows us to pass in parameters to the execution of our scripts. We supply these parameter values during deploy time, and these values are used to convey the name or location of various external resources. The scripts that install our applications can access these values and ensure the application is properly integrated with the set of resources not managed by the appliance.
What is your team looking to do next with WebSphere CloudBurst and their private cloud?
The next challenge on our plate is to keep up with the demand of our expanding cloud and to develop a more dynamic relationship between our native pools and our cloud using the Tivoli Provisioning Manager. These are fun challenges to have and we look forward to sharing our progress.
I'm glad I got to spend some time with Robbie to glean some insight into their work and progress with WebSphere CloudBurst. I hope this information was useful to you. It's always nice to hear about a product from practitioners who can give you hints, tips, gotchas, and other useful information. Be sure to let me know if you have any questions about what Robbie and his team are doing with WebSphere CloudBurst.