It's really hard to complain about my work week right now. As I write this blog, I'm sitting in the Congress Center in Düsseldorf, Germany looking out over the Rhine River. As an aside, in Germany it is the Rhein River, and I have a historical connection to this body of water. My surname, Amrhein, translates (loosely) to 'on the Rhein'. It does not take an expert in genealogy to conclude that I have ancestors who at one time or another lived very close to this important German waterway.
Okay, putting the family tree aside for a minute, there is a good reason that I am in Düsseldorf this week. The city, and specifically the Congress Center, is playing host to the IBM European WebSphere Technical Conference. I am here presenting sessions that include a WebSphere CloudBurst overview, a WebSphere CloudBurst hands-on lab, and an up-close look at one of our internal team's use of the appliance. I have done each of these sessions once so far, and attendance was great, audience participation high, and feedback forthcoming. I am hearing and seeing the same thing in other sessions, which is of course, ideal for us presenters.
Now, to focus in on WebSphere CloudBurst for a bit, it seems that I am hearing a recurring question this week from the mostly European audience: "Why is WebSphere CloudBurst delivered as an appliance?" I am sure that I addressed this question in a previous blog post, but I believe it bears revisiting. There are various reasons I could give for the appliance form factor, but I like to distill all of that down into three major reasons: Consumability, Performance, and Security.
From a solution consumability perspective, nothing beats the appliance approach. WebSphere CloudBurst is an integrated hardware and software solution that delivers a specific set of function. You do not have to install software, procure and maintain storage for resources on the appliance (images, patterns, scripts, etc.), and maintain software components over time. You simply drop the appliance in to your data center, perform a one-time initialization, hook it up to the network, and you are ready to start leveraging WebSphere CloudBurst to build out your private cloud. While there is definitely work to setup the cloud infrastructure that WebSphere CloudBurst deploys environments to, we can completely eliminate a significant portion of solution implementation lead time by delivering everything you need in the appliance.
The performance benefits of an appliance approach are a natural result of building an integrated hardware and software stack. Design and development teams provide optimizations in both the hardware and software based on the fact that both the hardware and software have intimate knowledge of each other's design. In other words, this is not a 'least common denominator' tuning approach. Rather, the integrated design leads to enhanced performance for the specific set of functionality provided by WebSphere CloudBurst.
Finally, appliances enable us to deliver a very hardened, secure device. We provide private key encryption of every resource stored on the appliance. That private key is unique to each appliance and cannot be modified. In addition, the physical casing is tamper-resistant. If someone removes the casing, a 'Get Smart' style kill switch puts the appliance in a dormant state. You must send the appliance to IBM so we can reset it before further use, thus providing an additional layer of physical protection on top of the encryption. These security features, plus more, like a shield that prevents anyone from executing code on the appliance, come right out of the box and require no end-user configuration activity. In this way, you can simply focus on leveraging the user security and access controls provided by WebSphere CloudBurst.
If you had any questions on the rationale behind the appliance form factor of WebSphere CloudBurst, I hope this helps. I am off for now... back to the conference and the wonderful city of Düsseldorf.
A while back I had a four part FAQ series inspired by questions arising from customer visits discussing the first release of WebSphere CloudBurst. With the recent release of WebSphere CloudBurst 2.0, I think it is a good time to revisit an FAQ series with an entirely new set of questions.
For the first part of the series, I want to address a question we get all the time now: "What is the difference between WebSphere CloudBurst and WebSphere Virtual Enterprise?" This question was always fairly common, but now even more so because the new Intelligent Management Pack option for WebSphere Application Server Hypervisor Edition allows you to deploy WebSphere Virtual Enterprise cells using WebSphere CloudBurst.
Fundamentally, the difference between the WebSphere CloudBurst Appliance and WebSphere Virtual Enterprise is a complementary one. WebSphere CloudBurst provides a means to create your application environments, deploy them into a shared, cloud environment, and then manage them over time. In this respect, the appliance focuses on bringing cloud-like capabilities to the application infrastructure layer of your application environments. WebSphere CloudBurst does give you management capabilities for your running, virtualized application environments (i.e. applying maintenances and fixes), but for the most part those capabilities do not extend into the application runtime environment.
Now, you may ask why WebSphere CloudBurst does not extend its reach into the application runtime. The answer is simple: We already have a solution that does just that, WebSphere Virtual Enterprise. WebSphere Virtual Enterprise provides capabilities that allow you to dynamically and autonomically manage your application runtime. You can use WebSphere Virtual Enterprise to not only assign performance goals to your applications, but also to declare the importance of a given application meeting its goals relative to other applications in your organization. This enables the dynamic management of your applications and their resources such that your applications perform according to their goals and relative importance to your business. Simply put, you get an elastic runtime at the application layer of your application environments.
As I said, WebSphere CloudBurst and WebSphere Virtual Enterprise are complementary solutions. Both enable notions of cloud computing, but at different layers of your application environments. WebSphere CloudBurst hones in on the application infrastructure components, while WebSphere Virtual Enterprise zeros in on the applications running in those environments. The new Intelligent Management Pack for WebSphere Application Server Hypervisor Edition means that WebSphere CloudBurst can now dispense WebSphere Virtual Enterprise environments into your on-premise cloud. That means you can take advantage of these complementary solutions from a single and simple management plane.
I hope this helps to clear things up. As always, questions and comments are welcome!
When I talk to users familiar with both WebSphere CloudBurst and the IBM Systems Director VMControl offering, there is sometimes a bit of confusion. It is not surprising. Both WebSphere CloudBurst and IBM Systems Director VMControl allow users to create and manage virtualized environments. That leads us to an oft-asked question: What is the difference between WebSphere CloudBurst and IBM Systems Director VMControl?
The simple answer is that the difference in the two offerings is the degree to which they are purpose-built. IBM Systems Director VMControl equips users with broadly applicable capabilities to create and manage environments consisting of virtual machines. These capabilities extend to PowerVM, z/VM, VMware, and Microsoft Hyper-V hypervisor platforms. IBM Systems Director VMControl is not necessarily knowledgeable about the software running in the virtual machine, but it does allow the user to manage that asset effectively.
Compare and contrast that with the capabilities provided by WebSphere CloudBurst. The appliance also enables users to create and manage environments consisting of virtual machines. The difference is that WebSphere CloudBurst is purpose-built to provide you with the ability to create, deploy, and manage virtualized WebSphere environments quickly and easily.
What does that mean? Well, on one hand it means that WebSphere CloudBurst does not treat the virtual machines it creates like a black box. In fact, it knows quite a bit about the software running inside those machines, and provides users with out-of-the-box configuration and administration capabilities for said software. WebSphere CloudBurst knows how to interact with the software in the virtual machines to do things like federate WebSphere nodes into a cell, create application server clusters, configure environments for optimal performance, apply fixes and upgrades, and more. The best part is you do not need to supply any of your own scripts to do this. In short, the appliance ships with WebSphere intelligence.
Beyond this WebSphere intelligence, WebSphere CloudBurst enables users to create customized WebSphere environments (from the operating system up) and codify those customized environments in the form of patterns. These patterns, which represent your very own WebSphere application environments, enable you to deploy your applications rapidly, repeatedly and with extremely consistent results. In addition, the appliance allows you to define varying roles for users, each of those mapping to traditional data center responsibilities (i.e. customizing the operating system, building application infrastructure, carrying out middleware customizations, etc.). Again, WebSphere CloudBurst was purpose-built with WebSphere environments in mind.
It is not all about comparing and contrasting WebSphere CloudBurst and IBM Systems Director VMControl. In the case that you are using WebSphere CloudBurst to create and manage virtualized WebSphere environments on top of the PowerVM hypervisor platform, IBM Systems Director VMControl is actually a required component. In this scenario, the two offerings are complementary. WebSphere CloudBurst communicates with IBM Systems Director VMControl in order to create and configure the virtualized WebSphere environment requested by the user. This image below depicts how the two products work in conjunction in a PowerVM environment.
I hope this helps to shed light on how WebSphere CloudBurst compares to, contrasts with, and complements IBM Systems Director VMControl. Feel free to reach out to me on the blog or on Twitter (@damrhein) with any questions I did not answer here.
One of the most powerful features of WebSphere CloudBurst is the ability to take one of the WebSphere Application Server Hypervisor Edition virtual images that are shipped with the appliance and extend it to a produce a custom virtual image. This allows users to begin creating customized environments from the bottom up, starting with the operating system. To put it in better context, let's take a look at a couple of scenarios where this feature comes in quite handy.
First off, a very common need for our customers is the ability to continually monitor their application environments. For instance, you may install Tivoli monitoring agents on all of your machines hosting WebSphere Application Server processes and configure those agents to report back to a management server. This is a great case for image extension in WebSphere CloudBurst.
In this scenario, you would start by extending an existing WebSphere Application Server Hypervisor Edition image. WebSphere CloudBurst creates a running virtual machine based off of the selected image, and you log into that virtual machine and install the Tivoli monitoring agents. Once the installation is done, you capture the virtual image back into the WebSphere CloudBurst catalog and use the new image to build a custom pattern. The last step is to include a script package on this custom pattern that, upon deployment, will configure the installed monitoring agents to report back to your desired management server.
Another use case is likely to be of interest to you if you are using WebSphere Virtual Enterprise (or something similar), and you could benefit from the same ease of provisioning for those environments that WebSphere CloudBurst brings to WebSphere Application Server environments. You can use the same customization combination above (image extension and custom scripts) to enable WebSphere CloudBurst to essentially dispense WebSphere Virtual Enterprise cells.
Again, this scenario starts off by extending a WebSphere Application Server Hypervisor Edition virtual image. Once the virtual machine for the extension is created by WebSphere CloudBurst, you log in and install the WebSphere Virtual Enterprise product. After the installation is done, you capture the image and store it in the catalog. Next, you build a custom pattern based off of this image and include script packages that, upon deployment, augment the various parts in the pattern from WebSphere Application Server profiles to WebSphere Virtual Enterprise profiles. (You may wonder why you wouldn't just create the WebSphere Virtual Enterprise profiles during the image extension process. This is because during image extension, you cannot make changes to the virtual disk that contains the WebSphere Application Server profiles. Any changes made to the profiles will be wiped out during the capture process.)
There are countless more scenarios for creating custom virtual images in WebSphere CloudBurst. To name a few, you may want to install JDBC drivers that are common to almost all of your application environments, install required anti-virus software, or just make operating system configuration changes. All of these things can be accomplished through the image extension and capture process. Look for an article coming out soon that will discuss and explain, in much greater detail than I provided here, the process of installing and configuring Tivoli monitoring agents in environments dispensed by WebSphere CloudBurst. In the meantime, if you have any questions or comments, drop us a line here or check out our forum.
I’m going to take a different approach this week in the blog. Instead of me telling you about some of the features or uses of WebSphere CloudBurst, I thought I would catch up with someone using the product everyday, WebSphere Test Architect Robbie Minshall. Robbie is responsible for a team of testers that harness a lab of over 2,000 physical machines to put our WebSphere Application Server product through some pretty rigorous testing. Toward the beginning of this year Robbie’s team started to leverage the WebSphere CloudBurst Appliance in order to create the WebSphere Application Server environments needed for their testing.
Robbie, can you tell us a little bit about what the WebSphere Application Server test efforts entail?
In WebSphere Application Server development and test we have two primary scenarios. The first is making sure that developers have rapid access to code, test cases and server topologies so that they can write code, test cases and then execute test scenarios on meaningful topologies. The second scenario is an automated daily regression where in response to a build, we provision a massive amount of WebSphere Application Server topologies and execute our automated regression tests.
Previously we have supported these scenarios through the deployment of the Tivoli Provisioning Manager for operating system provisioning, some applications for checking out environments, and then a lot of automation scripts for the silent install and configuration of WebSphere Application Server cells.
Given those scenarios and the existing solution, what are your motivations for setting up a private cloud using WebSphere CloudBurst Appliance?
We are supporting these scenarios through a pretty complicated combination of technologies. These include silent WAS install scripts, wsadmin configuration scripts, a custom hardware leasing application and the utilization of Tivoli Provisioning Manager for OS Provisioning. This solution is working very well for us though as always we are looking for areas to improve, opportunities to simplify and to reduce our dependency on investment in our custom automation scripts. Mainly, there were 3 areas where we wanted to improve our framework: Availability, Utilization and Management. This is why we started looking to the WebSphere CloudBurst Appliance.
Can you expand a bit on what you are looking for in those three areas?
The first focus area we have is availability of environments. We really wanted to lower the entry requirement for the skills and education necessary to get a development or test environment. Setting up these environments has just been too hard, too time consuming, and too error prone. Using WebSphere CloudBurst we can provide an easy push button solution for developers to get on-demand access to the topologies they need.
The second area we are looking for significant improvements on is hardware utilization. Our budgets are tight and in our native automation pools we are only using between 6-12% of the available physical resources. In order to improve this we were looking at leveraging virtualization. WebSphere CloudBurst offers the classic benefit of virtualization with the nice additions of optimized WebSphere Application Server placement and really good topology and pattern management. In our initial experiments we were able to push the hardware utilization up to 90% of physical capacity and consistently were leveraging around 70% of our physical capacity.
Finally we are looking to improve and simplify our management of physical resources and automation. We work in a lot of small agile teams and organizational priorities change from iteration to iteration. Not only does WebSphere CloudBurst allow us to maintain a catalog of topologies or patterns for releases but it also allows us to adjust physical resource allocation to teams through the use of sub clouds or cloud groups.
Basically we felt that WebSphere CloudBurst would improve the availability of application environments, enhance automation, and improve hardware utilization all with very low physical and administrative costs.
What were some of the challenges involved with getting a cloud up and running in your test department?
One of our challenges seems like it would be common to many scenarios, especially in today’s world. Our budget for new hardware to build out our cloud infrastructure was initially very limited. Most cloud infrastructure designs depict very ideal hardware scenarios including SANs, large multicore machines, and private and public networks within a dedicated lab. Quite frankly we did not have the budget to create this from scratch. It was important for us to demonstrate value and data to warrant future investment in dedicated infrastructure. After some performance comparisons we were very happily surprised to see that we could leverage our existing mixed hardware within a distributed cloud. The performance of application environments dispensed by WebSphere CloudBurst on many small existing boxes in comparison to large multicore machines with a SAN was very comparable. This allows us to leverage existing hardware, with minimal investment all the while demonstrating the value and efficiencies of cloud computing. That data in turn has allowed us to obtain new dedicated hardware to iteratively build up a larger lab specifically for use with WebSphere CloudBurst.
Specifically with WebSphere CloudBurst, are there any tips/hints you would offer users getting started with the appliance?
Sure. First, we quickly realized as we added hypervisors to our WebSphere CloudBurst setup it was critical to have someone with network knowledge on hand. This is because the hypervisors came from various sections of our lab, and we really needed people with knowledge of how the network operated in those different sections. Once we had the right people we were able to setup WebSphere CloudBurst and deploy patterns within an hour and a half.
Moving forward we continued to have challenges as we dynamically moved systems between our native hardware pool and our cloud. Occasionally the WebSphere CloudBurst administrator would move a system into the cloud but incorrectly configure the network or storage information. This lead to some misconfigured hypervisors polluting our cloud. We overcame this, quite simply and satisfactorily I may add, by creating some simple WebSphere CloudBurst CLI scripts which add the hypervisors, test them individually, by carrying out a small deployment to that hypervisor, and then move the correctly configured hypervisors into the cloud after verifying success. Misconfigured hypervisors go into a pool for problem determination. This has allowed us to maintain a clean cloud, and we are able to dynamically move our hardware in and out of the cloud to meet our business objectives.
We also use the WebSphere CloudBurst CLI to prime the cloud so to speak. Before using a given hypervisor in our cloud, we execute scripts that ensure each unique virtual image in our catalog has been deployed to each of our hypervisors at least once. When the image is first deployed to a hypervisor, a cache is created on the hypervisor side of the connection, thus meaning subsequent deployments do not require the entire image to be transferred over the wire. This gives us consistent and fast deployment times once we are using a hypervisor in our cloud.
I would assume that like many applications deployed on WebSphere Application Server, your team’s applications have several external dependencies. Some of these dependencies won’t necessarily be in the cloud, so how did you handle this?
You’re right about the external dependencies. Our applications and test cases run on the WebSphere Application Server but are dependent upon many external resources such as databases, LDAP servers, external web services etc. WebSphere CloudBurst allows us to deploy WAS topologies in a very dynamic and configurable way but the 1.0.1 version does not allow us to deploy these external resources in the same manner. This was overcome by using script packages in our patterns. These script packages allow us to associate our test applications with various patterns we have defined. The script package definition also allows us to pass in parameters to the execution of our scripts. We supply these parameter values during deploy time, and these values are used to convey the name or location of various external resources. The scripts that install our applications can access these values and ensure the application is properly integrated with the set of resources not managed by the appliance.
What is your team looking to do next with WebSphere CloudBurst and their private cloud?
The next challenge on our plate is to keep up with the demand of our expanding cloud and to develop a more dynamic relationship between our native pools and our cloud using the Tivoli Provisioning Manager. These are fun challenges to have and we look forward to sharing our progress.
I'm glad I got to spend some time with Robbie to glean some insight into their work and progress with WebSphere CloudBurst. I hope this information was useful to you. It's always nice to hear about a product from practitioners who can give you hints, tips, gotchas, and other useful information. Be sure to let me know if you have any questions about what Robbie and his team are doing with WebSphere CloudBurst.
Over the past several months industry focus on cloud computing seems to have only intensified. Within IBM and for the purposes of this blog, WebSphere, there have been several announcements and offerings that indicate our commitment and belief in the cloud computing approach.
To further highlight WebSphere's focus and offerings in the cloud computing realm, we are embarking on a "WebSphere in the Clouds" campaign during the months of September and October. Our intent is to virtually deliver information about our cloud strategy and offerings directly from the experts to you, our WebSphere users.
The event will be kicked off by WebSphere's Director of Product Management, Kareem Yusuf, on September 23rd from 9-10 EDT. Kareem will talk about cloud computing in the enterprise, and its unique relationship to SOA thoughts and principles. In addition, he'll give an overview of what WebSphere has been doing in the cloud computing space. This will be followed by sessions from technical experts that detail WebSphere offerings in both the public and private clouds, as well as sessions that discuss enablers of application and application infrastructure elasticity.
To find out more about the "WebSphere in the Clouds" campaign, you can check out the main announcement page. To sign up for the series of virtual events visit the registration page. We hope you will join us for the series of webcasts to learn all about WebSphere's work in the clouds.
Many technologies and ideas are paving the way for cloud computing. Utility computing, grid computing, and virtualization have all played important roles in enabling cloud solutions to take hold. In some ways, SOA is an easy to overlook player in the cloud computing world. However, there's no doubt that without SOA, and the ideas from the SOA movement, cloud computing would not be where it is now.
First, consider the millions of services available in the application services layer of the public cloud. While some of these services are intended to be consumed by an end-user, just as many are meant to be consumed programmatically. Enterprises seek to compose services in the application services layer to deliver larger, end-user applications to their consumers. As such, the ability to consume services that exist across domains and company firewalls is a must. SOA standards help in this respect as they define how services, regardless of location, are discovered, consumed, and governed. This common set of standards has helped to make the services in a public cloud more readily useable by enterprises, so SOA standards have been a key factor in the explosion of service offerings in the public cloud.
Second, and just as important, is the impact that SOA has and will continue to have on the enabling layers of cloud computing. By the enabling layers of the cloud, I mean the platform and infrastructure services layer where we find both application and physical infrastructure. These two layers in the cloud are often referred to as constituting a Service Oriented Infrastructure, so the impact of SOA is immediately obvious. SOA has led to viewing application and physical infrastructure capabilities as discrete services that can be consumed as part of an overall solution or process. As the number of services in these two layers continues to grow, it will be important that they can be discovered, managed, and governed similar to software service components so as to enable robust, composable cloud infrastructure solutions. By applying the principles and lessons of SOA to these enabling services, we can achieve a discoverable, composable, and governable cloud infrastructure.
SOA should be acknowledged as a key enabler to cloud computing solutions. There are of course reasons beyond what is mentioned above. For instance, think about application virtualization and how effective management of such virtualization requires the capability to interact with applications implemented in various technologies. SOA standards have established how to interact and communicate with applications regardless of implementation, so virtualization management can and should piggyback on these standards. As cloud computing continues to evolve, I think we will only see more instances of SOA affecting cloud computing for the better.
In the previous post Dustin shared a great video demonstrating the value of the IBM Image Construction and Composition Tool that is now delivered with IBM Workload Deployer V3.1. This is certainly one of the key new features of IBM Workload Deployer V3.1. However, there are also a number of other compelling enhancements and features that we would like communicate.
I created the attached video to highlight some of these features included in new Workload Deployer release. The video uses the web console to highlight some of the features and capabilities, giving a brief introduction for each one. Without going into a lot of depth, I think it gives a nice overview. This may be especially helpful if you already have Workload Deployer v3.0 and want to see the value you will get when you upgrade to Workload Deployer v3.1. Check it out.
We believe that these new features make IBM Workload Deployer V3.1 an even better solution for your private cloud needs. Please let us know what you think.
Lately Joe and I have been pretty vocal about bringing up the new IBM Image Construction and Composition Tool capabilities in IBM Workload Deployer v3.1. While writing about such new capabilities is always good, I think seeing is believing. In that light, I hope you will take a look at the recent demo I put together that shows how to use the Image Construction and Composition Tool with IBM Workload Deployer v3.1!
When IBM Workload Deployer v3.0 rolled around, the appliance introduced the concept of shared services. These were services that a cloud administrator could launch into the cloud infrastructure defined to IBM Workload Deployer, and use to serve a number of different application deployments. There were, and continue to be, two main shared services: a proxy service and a cache service. The shared proxy service does pretty much what you may guess. It provides request routing capabilities across multiple different instances of multiple different applications, thereby providing a centralized resource that encapsulates this basic need in an application environment. You can probably also guess what the caching service does. It caches things! Specifically, in IBM Workload Deployer v3.0 it provided in-memory caching of HTTP sessions, thus ensuring high availability of data stored in those sessions.
Undoubtedly, the ability to make HTTP session data fault tolerant is extremely critical in any application environment, cloud-based environments included. However, the applicability of a shared cache service is much further reaching, and in IBM Workload Deployer v3.1, we are starting to open this service up to your applications. What does this mean to you? Quite simply it now means that you can access this cache directly from your application code. If you are familiar with WebSphere eXtreme Scale or the DataPower XC10 Caching Appliance, then you know exactly what I mean. You can use the WebSphere eXtreme Scale ObjectGrid API to insert, read, update, and delete entries that exist in the in-memory cache. The underlying cache technology is based on the same code that powers WebSphere eXtreme Scale and DataPower XC10, so you can be sure that your cache is scalable, fault tolerant, responsive, and otherwise able to meet the needs of your application.
As I hope you find to be the case with many IBM Workload Deployer capabilities, this is a superbly simple capability to leverage. When you deploy virtual application patterns based on the IBM Workload Deployer Pattern for Web Applications, the capability is simply there. The underlying runtime that is serving your application is automatically augmented with the capabilities necessary so that your applications can connect to and utilize the deployed caching service. It is also worth pointing out that you can utilize the caching capabilities provided by this shared service for applications and application infrastructure that you deploy via virtual system patterns as well. You can either choose to augment the WebSphere Application Server environment with the XC10 Feature Pack (a deploy-time option for virtual system patterns built on WebSphere Application Server Hypervisor Edition v8), or you can configure WebSphere Application Server as you always would when integrating with a WebSphere eXtreme Scale environment or a DataPower XC10 Appliance.
What's the real benefit to all of this you ask? Well, when you use the shared caching service, you get the benefits of a distributed, in-memory, extremely scalable cache without having to deal with too much setup or administration. You simply tell IBM Workload Deployer how many resources you want to dedicate to your cache, and deploy the shared service. IBM Workload Deployer takes care of the details, including scaling in and out the cache to meet the needs of the system. On top of all of this, there is also an option to configure 'Next to the Cloud' caching. If you currently own DataPower XC10 appliances, you can make those available to virtual application pattern deployments (this was already possible with virtual system patterns) by simply providing details of the location of the appliance collective in question.
Put simply, setting up, administering, and utilizing an object caching service for your applications has never been easier. Check it out and let us know what you think!