The reason I suggest the application proxy approach is twofold. First, it affords you the ability of having custom interactions with the REST API. For instance, you may insert logic into the server-side proxy code that returns only a subset of the JSON data contained in the response from the appliance. Alternatively, in an effort to reduce the chattiness on your client-side, you may join JSON data from multiple different REST requests to the appliance to fulfill a single client request. You may even decide to represent the data in an all together different format than JSON. All of these options and many more are available to you if you implement an application-based proxy to the REST API.
The second reason I suggest the application approach is that it is easier, and seemingly safer, to not deal with user passwords on the client-side. If you setup your application proxy, you can configure it to retrieve the appropriate password from a secure location (like an encoded file) based on information passed along in the request. This means the password information is only present in the request (in encoded form of course) from the application proxy to the WebSphere CloudBurst Appliance.
The good news about the application-based proxy approach is that it is simple to put in place. I composed one using the open source Apache Wink project. The Apache Wink project is an open source implementation of the JAX-RS specification (and then some), and it enables you to develop POJOs that are in turn exposed in a RESTful manner. In my case, I had a single resource POJO:
The Apache Wink runtime routes any HTTP GET request whose path is like /resources/* to the getResources method in the WCAResource class. This method passes along information taken from the query string (the host name of the target WebSphere CloudBurst Appliance and the requesting WebSphere CloudBurst username), as well as the HTTP path information and sends it on to the getResource method declared as follows:
The getResource method above uses the WebSphere CloudBurst host name and the request path to construct the URL for the corresponding WebSphere CloudBurst REST API call. Next, it constructs an Apache Wink Resource object and sends the REST request along to the WebSphere CloudBurst Appliance. How do we authenticate this request? We use the WebSphere CloudBurst username (sent as a query string parameter) to retrieve the appropriate encoded password information. Once we have that, we construct the necessary header for basic authorization over SSL.
The application-based proxy shown here is simply a pass-through. It does not manipulate the data returned from the WebSphere CloudBurst REST API, nor does it map a single client-side call to multiple REST requests. However, it would be simple enough to extend it to do any of those things. If you have any questions about the code here, please let me know. I'd be happy to share more of the code, or talk about how and where to extend it.
Recently, IBM has made its presence in the cloud computing market known with a series of offerings and partnerships that position Big Blue nicely. There have been announcements of university partnerships, new cloud services and clients, and intent to deliver IBM software with Amazon Web Services. To further cloud computing and IBM’s offerings in cloud computing, teams of technical evangelists have been formed to spread the good news. I have joined one of these teams, and I’ll be here from time to time to talk about IBM’s work in the clouds.
Since we are just getting started, I figure it’s appropriate to touch on the definition and composition of cloud computing. I have read and heard hundreds of definitions for cloud computing, and they all make good points. Nearly every single definition describes a computing solution in which resources, both hardware and software, scale up and down to meet the needs of the cloud consumer. That consumer may be an end-user accessing applications that run in the cloud, or it may be the application running in the cloud that depends on the lower layer services of the cloud. Most of the existing definitions also imply some autonomic capability in which not only does the cloud scale up and down, but it does so without administrator intervention based on policies declared by the consumer. Personally, I like many of these aspects, so I have tried to combine the elements that I think are most important: Cloud computing provides computing resources in a scalable, autonomic, governable fashion. These resources may be software, application infrastructure, or physical infrastructure, and the overall solution enables IT to be delivered as a service.
Attempting to define the anatomy of cloud computing seems to elicit as many opinions as defining cloud computing. For me, the three-layer approach sums it up quite nicely. While it’s true that some cloud solutions span multiple layers, the Google App Engine comes to mind, it provides at least a reference point for the discussion of cloud products.
Application Services: This layer is comprised of what we have come to know as Software as a Service. This layer is very familiar to us (GMail, Facebook, MySpace, etc.), and it is equally familiar with enterprise consumers (WebSphere sMash on EC2, Salesforce, Sugar CRM, etc.).
Platform Services: The platform services layer is made up of different services that support applications. This may include middleware, connectivity, data, and messaging services. Offerings, such as WebSphere Application Server Virtual Images, SimpleDB on AWS, and Memcache from Google are all good examples of platform services.
Infrastructure Services: Infrastructure services provide physical resources as needed. These include hardware, networking, storage, and more. IBM’s Blue Cloud, Amazon’s EC2, and Google App Engine are examples of infrastructure service providers.
Looking at all three layers, it’s plain to see that starting with application services, each layer builds on the other. However, that does not mean that each layer cannot be used independently of the other. In fact, companies often construct on-ramp paths to cloud computing that start with services in only one of the layers (i.e. virtualization of hardware).
So, there's my shot at defining cloud computing! To be sure, my view of the cloud has evolved over time. The more opinions and thoughts I read, the more I challenge my own view. For that reason, I’d like to hear what you think. What is the definition and anatomy of your cloud?
I was very encouraged by the consistently positive response we got at IMPACT for our WebSphere CloudBurst and Rational Automation Framework for WebSphere (RAFW) integration. I believe there were many reasons for this response: accelerated time to value, decreased investment needs for activities that are not core to your business, lowered barrier of entry for provisioning and configuring WebSphere cells, and much more. While those are certainly all very real and valuable benefits, I also believe that quite a bit of interest in this integrated solution comes from the fact that it is applicable to a number of needs common to you, our WebSphere users.
With that in mind, let's look at some (not all) of the scenarios where WebSphere CloudBurst and RAFW integration can help you:
Create WebSphere CloudBurst patterns that include configuration without scripting: Users love our WebSphere CloudBurst patterns. They really see the value in codifying both the topology and configuration of their application infrastructure. However, some users do not have existing WebSphere configuration scripts and do not have the time and/or resource to invest in creating these scripts. They are looking for a solution that provides not only the provisioning of WebSphere environments but also the configuration of said environments (configuration beyond what our IBM Hypervisor Edition images encapsulate, specifically application-oriented configuration). RAFW provides this capability in the form of 500+ out-of-the-box configuration actions for WebSphere environments. This includes actions to create JDBC resources, create JMS queues, deploy applications, configure application containers, and much, much more. You can create WebSphere CloudBurst patterns that contain a special script package, which points back to a RAFW project containing a set of configuration actions. During deployment, WebSphere CloudBurst will provision your WebSphere environment and then cause the invocation of the specified RAFW project, which in turn runs a set of configuration actions against the provisioned environment. This means you can set up full-blown, ready-to-go application environments with absolutely no user-supplied scripting. In fact, I took this approach to setup a J2EE performance benchmark application, DayTrader 2.0, running on WebSphere Application Server. Those of you familiar with the application know this is not a trivial environment to stand up. Yet, I did it without having to personally write a single line of configuration scripting, and it was all ready to go in around thirty minutes.
Creating WebSphere CloudBurst patterns from existing environments: This comes up all the time. I go through a standard introduction to WebSphere CloudBurst, users see the value, love the patterns-based approach, and immediately want to know how they get their existing environments into the form of a pattern. RAFW, along with the special WebSphere CloudBurst script package, can make this a straightforward and hardened process. You use a capability in RAFW to import the configuration of an existing cell, thereby creating a RAFW environment for that configuration. You then create a WebSphere CloudBurst pattern with a topology congruent to your existing environment, attach the special script package I mentioned earlier, and you are done with the import! When you deploy this pattern, you simply specify the RAFW environment that you created earlier (the one that holds the configuration data for your existing environment) and a RAFW project that will apply the configuration data in that RAFW environment to the WebSphere environment provisioned by WebSphere CloudBurst. The creation of the WebSphere environment, as well as its configuration, happens in a completely automated fashion.
Configure, capture, reuse: There are many situations that may require you to make manual changes to a WebSphere cell after it has been deployed. For example, during performance testing for your application, you might discover that you need to tweak the number of available threads in the web container. As another example, for the first setup of a given application environment, you may want to quickly deploy the cell using WebSphere CloudBurst and then manually install and configure your applications to make sure everything is just right. In either case, it is likely that you want to capture the updated configuration and make sure that any future deployments use those updates. Again, WebSphere CloudBurst and RAFW makes this simple. First, you build a pattern that encapsulates your WebSphere topology (the types and quantity of nodes you want) and attach the special script package mentioned above. For the first deployment, you simply specify the name of the new RAFW environment you want to create. Once the system is up, you log into the WebSphere administration console, make your necessary customizations, and then you use RAFW to import that updated configuration thus updating the initially created RAFW environment. For subsequent deployments, you simply deploy the same pattern, specifying the same RAFW environment as well as a RAFW project, which RAFW automatically created for you during the first deployment. This project applies the configuration (the one you manually established and imported into RAFW) to the WebSphere environment setup by WebSphere CloudBurst.
Configure WebSphere environments across virtual and physical settings: It seems that in many cases our users manage the same WebSphere environment across both virtual and physical settings. For example, they may provision the application environment using WebSphere CloudBurst for everything from development to pre-production, and then for production provision that same environment to a set of physical servers. At least, they try to provision the same environment. In reality, it is tough to reproduce the exact same configuration once you break from the WebSphere CloudBurst patterns-based approach. However, if you stored the configuration of your WebSphere cell as a RAFW environment, you could apply that configuration data to a WebSphere cell regardless of whether it existed in the physical or virtual world. Once you move to physical, you do lose out on the fast provisioning, WebSphere intelligence, cloud management capabilities, and automated integration with RAFW that you get when using WebSphere CloudBurst, but if it is in your process to move to physical hardware at some point, reusing the same RAFW environment certainly eases the migration task.
I hope this sheds some light on some of the common issues WebSphere CloudBurst and the Rational Automation Framework for WebSphere can combine to solve really well. This is by no means an exhaustive list, but really meant to point out the broad application of the solution. If you want to see how it works, check out this video.
As I have mentioned before, IBM Workload Deployer v3.0 introduces choices in pattern-based deployment models. One of those models, virtual system patterns, is a carry over from the WebSphere CloudBurst Appliance. When you use virtual system patterns in IBM Workload Deployer, you can take advantage of all of the techniques you put to use in WebSphere CloudBurst. This is certainly good news for current WebSphere CloudBurst users, but it goes a bit further. Instead of simply maintaining the status quo with virtual system patterns, which would have been reasonable considering the introduction of virtual application patterns, we chose to continue to expand on your customization options for this pattern deployment model. In particular, I want to discuss three new features in IBM Workload Deployer that may help you to better construct and manage virtual system patterns.
The first new feature is one that I have been eagerly awaiting. In the new version of the appliance, we provide you with the ability to specify part and script package ordering in your pattern. This means that, within the virtual system pattern editor, you can tell IBM Workload Deployer in which order to start the virtual machines in your pattern, and you can specify in which order to invoke the script packages within the pattern during deployment. This eliminates the need for special script invocation orchestration logic in your pattern (I had customers resorting to a semaphore like approach using a shared file system), and it allows you to be more declarative about the virtual machine bring-up process. There are constraints, specifically with the part ordering. Some images will impose an implied part start-up order that you cannot change. For instance, deployment manager parts in the WebSphere Application Server Hypervisor Edition image must start before custom node parts. The good news is the pattern editor will not allow you to specify a part start-up order that violates these constraints. The image below shows an example of the ordering view in the virtual system pattern editor.
Another new feature that may influence the way you build virtual system patterns is the introduction of Add-Ons. You can think of Add-Ons as special script packages that you can include in your virtual system pattern that perform system-level configuration actions. Specifically, you can include add-ons in your virtual system pattern to add an operating system user, add a virtual disk, or add a NIC during the deployment process. You include Add-Ons in your pattern by simply dragging and dropping them onto a part in your pattern, just as you do with script packages today. The difference between script packages and Add-Ons is that IBM Workload Deployer will ensure the invocation of all Add-Ons before any other scripts run during deployment. We include default Add-On implementations for adding a user, disk, and NIC.
The last new feature I want to talk about today has more to do with how you manage or govern the deployment of virtual system patterns. In WebSphere CloudBurst 2.0, we introduced the idea of Environment Profiles as a way to extend your customization reach into the deployment process. Initially, these profiles gave you the ability to directly assign IP addresses to virtual machines in your deployment, declaratively specify virtual machine naming formats, and easily split a single pattern deployment across multiple cloud groups. In IBM Workload Deployer, you will be able to use these same profiles to set resource consumption limits for pattern deployments. In particular, you will be able to set cumulative limits for virtual CPU, memory, storage, and software licenses used by deployments tied to a specific profile, thereby giving you finer-grained control over cloud resource consumption. The picture below shows the new resource limit aspects of environment profiles.
Virtual system patterns are key in the deployment model choices for IBM Workload Deployer. Not only did we carry the concept over from WebSphere CloudBurst to IBM Workload Deployer, but we made it even better. Expect this trend to continue!
A while back I co-authored an article along with Chris Ahl from Tivoli and Ken Klingensmith from WebSphere Technical Sales about the customization of virtual images in WebSphere CloudBurst. In the article we approached image customization as a means to enable IBM Tivoli Monitoring for the operating system within virtual machines dispensed by WebSphere CloudBurst. Today I posted a short demonstration that discusses and shows this particular integration scenario. If you are interested, but haven't had time to read the article, you may want to watch the video first as it should give you a good overview of the process and results.
Talk of Tivoli reminds me that IBM Pulse 2010 is just around the corner. I'll be going to discuss WebSphere CloudBurst and how it can be paired with software from IBM Tivoli for high-value integration scenarios. In the session I'll be talking about the Tivoli Monitoring integration as well as other key points such as our integration with Tivoli Service Automation Manager, IBM CloudBurst, and more. The best part about the session is that I will be co-presenting alongside a WebSphere CloudBurst customer that will dole out practical advice for using WebSphere CloudBurst within the enterprise. Join us on Tuesday February 23rd from 3:30 - 4:30 in Conference Center 306.
Remember, any time you have questions about WebSphere CloudBurst please pass them along. You can leave comments on this blog, or you can reach me at my new Twitter location @damrhein.
When it comes to administration of WebSphere environments, I (and many others) am a big fan of scripting. In my view, any administrative action you carry out with frequency > 1 is ideally suited for a script. The downside to not using scripts (longer configuration times, inconsistent configurations, isolated expertise) is simply too steep in most cases. I also realize that simply saying that you should script is not enough. For some, the learning curve can be a bit daunting. Quite frequently, I talk about our samples gallery or provide posts with embedded scripts in the hopes that I can help flatten out this curve a bit.
While these samples can certainly help to speed up your scripting efforts for certain use cases, they are more or less helpful for solving tactical challenges when scripting. If you and your company are embarking down a strategic path that includes beefing up your administrative scripting capability, I would strongly suggest you look at a resource a few of my colleagues pointed me at recently.
The resource I am talking about is the wsadminlib.py package referenced here. This python script file is a collection of hundreds of methods that carry out common WebSphere Application Server administrative tasks. The authors carefully constructed these methods with clear method and parameter names. The result is a script resource that can become the foundation for your custom-crafted administrative scripts.
I recently downloaded the wsadminlib.py script and began constructing WebSphere CloudBurst script packages to utilize it. To say I am impressed would be an understatement. This file makes so many tasks so incredibly simple. Take for instance the creation of an SIBus. That's just a simple call like the following:
wsadminlib.createSIBus(clusterName, nodeName, serverName, SIBusName, scope, secure)
How about associating a shared library with an application or application module? Another one-line call:
wsadminlib.associateSharedLibrary (libName, appName, warName)
Or what about setting a custom property in the webcontainer? You guessed it. One-line:
wsadminlib.setWebContainerCustomProperty(nodeName, serverName, propName, propValue
This is just an extremely small sample of what the wsadminlib.py includes. As I mentioned earlier, there are hundreds of other methods that carry out various tasks including: installing applications, creating core groups, creating virtual hosts, installing BLAs, creating JMS queues, and much more. If you are looking to beef up your WebSphere Application Server scripting efforts, or if you are just starting, I strongly encourage you to look into and make use of this valuable resource!
I’m going to take a different approach this week in the blog. Instead of me telling you about some of the features or uses of WebSphere CloudBurst, I thought I would catch up with someone using the product everyday, WebSphere Test Architect Robbie Minshall. Robbie is responsible for a team of testers that harness a lab of over 2,000 physical machines to put our WebSphere Application Server product through some pretty rigorous testing. Toward the beginning of this year Robbie’s team started to leverage the WebSphere CloudBurst Appliance in order to create the WebSphere Application Server environments needed for their testing.
Robbie, can you tell us a little bit about what the WebSphere Application Server test efforts entail?
In WebSphere Application Server development and test we have two primary scenarios. The first is making sure that developers have rapid access to code, test cases and server topologies so that they can write code, test cases and then execute test scenarios on meaningful topologies. The second scenario is an automated daily regression where in response to a build, we provision a massive amount of WebSphere Application Server topologies and execute our automated regression tests.
Previously we have supported these scenarios through the deployment of the Tivoli Provisioning Manager for operating system provisioning, some applications for checking out environments, and then a lot of automation scripts for the silent install and configuration of WebSphere Application Server cells.
Given those scenarios and the existing solution, what are your motivations for setting up a private cloud using WebSphere CloudBurst Appliance?
We are supporting these scenarios through a pretty complicated combination of technologies. These include silent WAS install scripts, wsadmin configuration scripts, a custom hardware leasing application and the utilization of Tivoli Provisioning Manager for OS Provisioning. This solution is working very well for us though as always we are looking for areas to improve, opportunities to simplify and to reduce our dependency on investment in our custom automation scripts. Mainly, there were 3 areas where we wanted to improve our framework: Availability, Utilization and Management. This is why we started looking to the WebSphere CloudBurst Appliance.
Can you expand a bit on what you are looking for in those three areas?
The first focus area we have is availability of environments. We really wanted to lower the entry requirement for the skills and education necessary to get a development or test environment. Setting up these environments has just been too hard, too time consuming, and too error prone. Using WebSphere CloudBurst we can provide an easy push button solution for developers to get on-demand access to the topologies they need.
The second area we are looking for significant improvements on is hardware utilization. Our budgets are tight and in our native automation pools we are only using between 6-12% of the available physical resources. In order to improve this we were looking at leveraging virtualization. WebSphere CloudBurst offers the classic benefit of virtualization with the nice additions of optimized WebSphere Application Server placement and really good topology and pattern management. In our initial experiments we were able to push the hardware utilization up to 90% of physical capacity and consistently were leveraging around 70% of our physical capacity.
Finally we are looking to improve and simplify our management of physical resources and automation. We work in a lot of small agile teams and organizational priorities change from iteration to iteration. Not only does WebSphere CloudBurst allow us to maintain a catalog of topologies or patterns for releases but it also allows us to adjust physical resource allocation to teams through the use of sub clouds or cloud groups.
Basically we felt that WebSphere CloudBurst would improve the availability of application environments, enhance automation, and improve hardware utilization all with very low physical and administrative costs.
What were some of the challenges involved with getting a cloud up and running in your test department?
One of our challenges seems like it would be common to many scenarios, especially in today’s world. Our budget for new hardware to build out our cloud infrastructure was initially very limited. Most cloud infrastructure designs depict very ideal hardware scenarios including SANs, large multicore machines, and private and public networks within a dedicated lab. Quite frankly we did not have the budget to create this from scratch. It was important for us to demonstrate value and data to warrant future investment in dedicated infrastructure. After some performance comparisons we were very happily surprised to see that we could leverage our existing mixed hardware within a distributed cloud. The performance of application environments dispensed by WebSphere CloudBurst on many small existing boxes in comparison to large multicore machines with a SAN was very comparable. This allows us to leverage existing hardware, with minimal investment all the while demonstrating the value and efficiencies of cloud computing. That data in turn has allowed us to obtain new dedicated hardware to iteratively build up a larger lab specifically for use with WebSphere CloudBurst.
Specifically with WebSphere CloudBurst, are there any tips/hints you would offer users getting started with the appliance?
Sure. First, we quickly realized as we added hypervisors to our WebSphere CloudBurst setup it was critical to have someone with network knowledge on hand. This is because the hypervisors came from various sections of our lab, and we really needed people with knowledge of how the network operated in those different sections. Once we had the right people we were able to setup WebSphere CloudBurst and deploy patterns within an hour and a half.
Moving forward we continued to have challenges as we dynamically moved systems between our native hardware pool and our cloud. Occasionally the WebSphere CloudBurst administrator would move a system into the cloud but incorrectly configure the network or storage information. This lead to some misconfigured hypervisors polluting our cloud. We overcame this, quite simply and satisfactorily I may add, by creating some simple WebSphere CloudBurst CLI scripts which add the hypervisors, test them individually, by carrying out a small deployment to that hypervisor, and then move the correctly configured hypervisors into the cloud after verifying success. Misconfigured hypervisors go into a pool for problem determination. This has allowed us to maintain a clean cloud, and we are able to dynamically move our hardware in and out of the cloud to meet our business objectives.
We also use the WebSphere CloudBurst CLI to prime the cloud so to speak. Before using a given hypervisor in our cloud, we execute scripts that ensure each unique virtual image in our catalog has been deployed to each of our hypervisors at least once. When the image is first deployed to a hypervisor, a cache is created on the hypervisor side of the connection, thus meaning subsequent deployments do not require the entire image to be transferred over the wire. This gives us consistent and fast deployment times once we are using a hypervisor in our cloud.
I would assume that like many applications deployed on WebSphere Application Server, your team’s applications have several external dependencies. Some of these dependencies won’t necessarily be in the cloud, so how did you handle this?
You’re right about the external dependencies. Our applications and test cases run on the WebSphere Application Server but are dependent upon many external resources such as databases, LDAP servers, external web services etc. WebSphere CloudBurst allows us to deploy WAS topologies in a very dynamic and configurable way but the 1.0.1 version does not allow us to deploy these external resources in the same manner. This was overcome by using script packages in our patterns. These script packages allow us to associate our test applications with various patterns we have defined. The script package definition also allows us to pass in parameters to the execution of our scripts. We supply these parameter values during deploy time, and these values are used to convey the name or location of various external resources. The scripts that install our applications can access these values and ensure the application is properly integrated with the set of resources not managed by the appliance.
What is your team looking to do next with WebSphere CloudBurst and their private cloud?
The next challenge on our plate is to keep up with the demand of our expanding cloud and to develop a more dynamic relationship between our native pools and our cloud using the Tivoli Provisioning Manager. These are fun challenges to have and we look forward to sharing our progress.
I'm glad I got to spend some time with Robbie to glean some insight into their work and progress with WebSphere CloudBurst. I hope this information was useful to you. It's always nice to hear about a product from practitioners who can give you hints, tips, gotchas, and other useful information. Be sure to let me know if you have any questions about what Robbie and his team are doing with WebSphere CloudBurst.
Lately, I have run into multiple situations where an IBM Workload Deployer user has been trying to decide exactly how they want to create their customized images for the cloud. Essentially, they have been trying to decide whether to use the native extend and capture capabilities of IBM Workload Deployer, or to pursue the use of the Image Construction and Composition Tool (also included with the appliance). The conversations have been interesting and challenging, but more importantly, they have been a reminder that constructing enterprise-ready environments for the cloud does not happen by magic. It takes thought, deliberate planning, sustainable design, and the tools to carry everything out.
The tools part we have covered. I have every confidence, bolstered by user experience after user experience, that IBM Workload Deployer and associated tools (like the Image Construction and Composition Tool) equip you to build highly customized, cloud-based application environments. In this post, I want to focus in on the thought process that goes into how you decide to build your customized environment. Specifically, I would like to talk about important points to consider as you try to understand whether to use the native extend and capture capabilities of IBM Workload Deployer or the Image Construction and Composition Tool.
To be clear from the outset, I am not trying to provide a decision flowchart in this post. For all intents and purposes, that would be next to impossible. Instead, I want to pose to you some important questions that you should ask of yourself, along with the reasons why I believe those queries to be important. Keeping in mind that this is not an all-inclusive list, here it goes:
Question: Are the customizations that you want to make congruent with an IBM-supplied image?
Reason: One of the first decisions you should make is whether or not you can start with an IBM-supplied image as the base for your customization. You need to know what middleware elements (type and version) make up your environment and what operating system should host that environment (version and distribution). You can match that information against the list of content that IBM supplies. If there is a match, you should start by looking at extend and capture to customize that image to meet your needs. If there is no direct match, you may be looking at the Image Construction and Composition Tool.
Question: Does your custom content supplement middleware content supplied in an IBM image?
Reason: If you simply need to add additional components that supplement software already in an IBM image, I believe it is best to first examine the use of extend and capture. Whether these components are IBM software or not is irrelevant as the extend and capture functionality does not care.
Question: How configurable do you want to make the custom content in your image?
Reason: If you are adding content into the image, you need to think about just how configurable you need it to be. When you use extend and capture, you add the content to an existing image in a manner that pretty well ends up being opaque to IBM Workload Deployer. To configure that content, you need to have script packages and make sure they are part of every pattern you create based on the image. Alternatively, if you use the Image Construction and Composition Tool, you can embed configuration behavior in the image's activation engine, and you can expose deploy-time parameters without needing to include script packages in every single pattern. As an example, if you need to add a monitoring agent into your environment, you would likely do this via extend and capture and end up with a pretty simple script package to configure that agent during deployment. If however, you need to create an image with a custom database, you would likely favor the Image Construction and Composition Tool as you could embed common deploy-time configuration parameters directly in the image. For a database, there are likely to be many more deploy-time configuration parameters that you want to expose as compared to a more simple monitoring agent.
Question: Is your main focus on making operating system changes?
Reason:If your primary focus is on making operating system changes AND the answer to the first question is that your target content aligns well with IBM-supplied images, then extend and capture is where you want to start. Of course, you need to make sure that you can make all necessary changes to the OS with extend and capture, but I will say that this capability is not very restrictive at all.
Admittedly, this is a short list, but I believe it is a good starting point for how you decide upon one approach versus the other. Also, I would be remiss not to point out that these tools are absolutely not mutually exclusive. Many users I work with use a combination of the two approaches. In fact, there are some use cases that call for both tools. Start by creating a completely custom image in the Image Construction and Composition Tool, and then subject that image to the extend and capture process in IBM Workload Deployer to customize it for a particular purpose, team, project, etc. I hope you find this helpful, and I welcome your feedback or thoughts!
At the core of cloud computing discussions and cloud computing in general is the idea of virtualization. The word 'virtualization' might invoke different things depending on who you talk to but for cloud discussions many people think of virtual images: entire systems being stored even down to the operating systems. The advantages seem evident. Instead of complex deployment models you simply need to take an unused piece of hardware resource and lay in the exact pattern. The assumption is that the hardware is free an compatible but everything else is negotiable. I think, however that there are different levels of assumption within the cloud concept. Laying in entire systems from the OS up may be way more work than is necessary. The advantage is that there are fewer constraints on what kinds of things you can depoy in your cloud. But one of the big disadvantages are that there are fewer constraints on what kinds of things you can depoy in your cloud. Sometimes, productivity is higher when your developers are given the topology parameters and when he knows what resources are going to be available. In fact, the concept of cloud seems to infer that there are fewer assumptions needed so you will have maximum versatility. However, flexibility is an antagonist to stability and stability is needed for prodcutivity. In effect, more assumptions necessarily equals faster developement and quicker time to release. So is cloud the antithesis of productivity? No, of course not. The beauty of clouds is that you can have as many assumptions as you want. A productive cloud model could assume specific hardware, OS and even webservers and macro-topologies. The cloud's resources could simply be avalable platforms that serve as quickly added nodes that can be dynamically provisioned within an appserver deployment. I think cloud models can be stratfied based on the number of assumptions that are built into it. Level 0 clouds could be where the only assumtion is the that the physical machines will support the virtual images. Level 6 could be that virtual servers and resources can be cloned by template to act as expansion nodes to meet growing demand. Does this cross over into other scaling models? Yes. So what? It doesn't have to be unique in every aspect it merely has to be consistent for effective use.
Sorry for the late notice - but I just realized that I hadn't blogged about a webcast that I am participating in tomorrow (Tuesday, 9/13)!
Chris Brealey (a Senior Technical Staff Member and Rational Enterprise Architect) and I are hosting an InformationWeek WebCast tomorrow (Tuesday, 9/13) entitled "Quickly and Efficiently Design, Develop, Deploy, and Test Workload Application Patterns to Save Months and Millions". I encourage you to register now for this free event (or if you can't make it tomorrow listen to it at your convenience as it will be recorded ... but you still need to register).
I'm really looking forward to this webcast. IBM Workload Deployer's predecessor, WebSphere Cloudburst Appliance, delivered unmatched capabilities for middleware deployments and management using Virtual System patterns (topology) - delivering complete middleware topologies in a rapid, consistent, and repeatable fashion. This has greatly improved the ability of development and test organizations to meet the ever increasing demands of today's agile development processes in addition to the assurance it provides for production environments. All of that value is still present (and improved) in IBM Workload Deployer but there is even more value in the new Virtual Application Patterns, as we've mentioned in previous posts.
Virtual Applications build upon this same notion of consistency and speed found in Virtual Systems while at the same time introducing a radical simplification to hosting your applications. Using an application-centric, declarative approach with Virtual Applications (workloads) it is even easier to deliver your applications rapidly leaving Workload Deployer to ensure the middleware environment is constructed and optimized to meet your application criteria. Virtual Applications usher IBM Workload Deployer into the realm of Platform-as-a-Service ... with even greater simplicity and agility to host your application in the most efficient fashion. As with Virtual System patterns earlier, we expect the introduction of Virtual Applications to continue to improve the dev/test lifecycle as well as production. The robust capabilities of Rational Application Developer and the simplicity of Virtual Application patterns in Workload Deployer make for a great combination.
I will start off the webcast with a discussion of PaaS and IBM Workload Deployer Virtual Application patterns. Chris will then discuss the application development process and how that is influenced with the introduction of the cloud environment. Chris will then explore the integration that is available in Rational Application Developer for IBM Workload Deployer. Finally, we will walk through a scenario that demonstrates how to leverage Virtual Application patterns in IBM Workload Deployer to design a solution that is then shared with the developer. Using Rational Application Developer the developer delivers the application into the pattern and moves it to test and finally pre-production. We will end with a question and answer time. I hope you can join us as we explore how we can use these technologies to increase agility and efficiency.
If you've read anything I've written about WebSphere CloudBurst up to this point you know all about patterns. Using the appliance you can easily and quickly build, deploy, and manage these representations of your middleware application environments. Today, I want to focus in on the deployment piece in particular and take a look at how you can easily automate this process.
You can use the WebSphere CloudBurst web console to deploy patterns, and when doing so you can even schedule the deployment to happen at a later date. This scheduling capability certainly gets you on the road to an automated deployment process, but what if you want to take it one step further and eliminate the need for someone to login and manually move around the web console to schedule automated deployments? In this case, you can use either the CLI or the REST interface that WebSphere CloudBurst offers.
In this post I thought I'd take a look at using the CLI interface in order to set the stage for some nice automation around pattern deployment. It starts out with a properties file that provides details about my deployment. This includes the cloud to deploy to, the pattern to deploy, password information, and the time at which the virtual system should start.
SYSTEM_NAME_PREFIX=New App Development
TARGET_CLOUD=Default ESX group
TARGET_PATTERN=WebSphere single server
Imagine that the properties file above gets written as the result of some other action, such as the completion of your application's build process. With the properties file in place, and I'll point out that your properties file can and probably will be more robust than above, let's move on to the code that handles the deployment process based on the information in said file. First, we have a small amount of CLI code to retrieve and parse the input data (I omitted the straight-forward properties retrieval for space):
from datetime import datetime, timedelta
from java.util import Properties
from java.io import FileInputStream
// read in and retrieve properties using java.util.Properties API (i.e. props.getProperty('DEPLOYMENT_DATE'))
parsedParts = deploymentDate.split(" ")
systemName = systemName + "_" + deploymentDate
dateParts = parsedParts.split("/")
timeParts = parsedParts.split(":")
monthPart = int(dateParts)
dayPart = int(dateParts)
yearPart = int(dateParts)
hourPart = int(timeParts)
minutePart = int(timeParts)
Next is the code that actually schedules the pattern deployment:
First we get the desired deployment time and current time as datetime objects. After that, assuming the desired deployment time has not already elapsed, we calculate the difference between the desired deployment time and current time. This difference, in seconds, is then added to the result of the time.time() value to come up with a start time. After that is done, we simply retrieve the cloud that was indicated in the properties file, and then we call the runInCloud method for the pattern indicated. When calling the runInCloud method we supply the name of the virtual system that will be created, password information, and the start time we calculated earlier. As a result of this method call, a task will be generated in the target WebSphere CloudBurst Appliance and the virtual system will be started at the specified time. This will happen in an automated fashion with no human intervention required.
That's really all there is to automating the pattern deployment process using the CLI. In a more complete, end-to-end scenario you may envision the completion of one process, such as an application build process mentioned above, result in the writing of the properties file and in turn the call into the CLI to deploy a pattern. As always, feel free to send me any comments or questions.
In WebSphere CloudBurst, a script package is your vehicle to provide custom middleware configuration. This may mean installing applications, configuring application dependencies, or otherwise tuning the middleware layer. Script packages are essentially ZIP files that include some executable (shell script, wsadmin script, Java program, etc.), and optionally, artifacts that support the execution of the script. As was the intention, you can achieve just about anything you want with a script package. This allows you to be as flexible and creative as you need to be, but it can also leave you asking "Where do I start?" In this post, I want to take an in-depth look at constructing and using a script package in WebSphere CloudBurst.
Specifically, I want to create a script package that supplies configuration functionality for something I believe a fair number of you do: change the default ports used in WebSphere Application Server. To create this and deploy a pattern using the script package, I do the following:
Create a shell script that configures the desired ports
Add the new script as a WebSphere CloudBurst script package
Create a pattern with the new script package
Deploy the pattern and verify the result
First things first. I create the following shell script that configures the ports:
The script uses documented ANT commands included with the WebSphere Application Server to update the ports based on a starting port number. You will notice the script first sources the /etc/virtualimage.properties file. This file is automatically created by WebSphere CloudBurst on every virtual machine it starts. The file is a key/value file with basic information about the WebSphere cell such as the install root ($WAS_INSTALL_ROOT), the profile name ($PROFILE_NAME), host name ($HOSTNAME), and more. For a full list of the data that WebSphere CloudBurst includes in this file, check out this documentation.
In addition to utilizing the standard set of variables provided by WebSphere CloudBurst, my script above also makes use of the $STARTING_PORT variable. Obviously this variable is not in the standard set. In fact, I define the STARTING_POINT variable when I define my new script package in WebSphere CloudBurst.
First I zip up the shell script above and attach it to the new script package. Next, I tell WebSphere CloudBurst where to unzip the script package on the virtual machine, how to invoke the included script, and the name of any parameters to associate with the script. Once that is done I can use the script package in a new pattern.
For the sake of simplicity here, I create a new pattern by cloning an existing WebSphere Application Server single server pattern. I drag and drop the new Configure ports script package on the single part and end up with the pattern shown below.
Now I am ready to deploy the pattern by clicking the Deploy button. During the deployment process I configure each part in the pattern (in this there is only a single part). I supply configuration information like virtual memory allocation, WebSphere cell name, WebSphere node name, and password information. In addition, I also supply a value for the STARTING_PORT parameter that is part of the Configure ports script package included in the pattern. The value I supply here will get inserted into the /etc/virtualimage.properties file on the virtual machine, and the value's key will be STARTING_PORT.
Once the configuration information is supplied, I click OK on the configuration panel and deployment panel, and WebSphere CloudBurst goes about standing up my virtualized WebSphere cell and running my script to configure the ports for the server instance. When it is done, I login to the WebSphere Application Server administration console to verify my results. To do this, I navigate to the configuration for the single application server instance, and pull up its port definitions.
Based on the results I can see my customizations took effect. I successfully captured my own unique WebSphere environment (in this case with a custom port range) in the form of a pattern. This custom environment can be deployed as many times as I need, in an automated fashion, and I'm guaranteed consistent results each and every time.
I hope this gives you a better idea of what script packages are all about and how they can utilize both WebSphere CloudBurst and user-supplied data that exists in the /etc/virtualimage.properties file of each virtual machine. If you have any questions let me know. I'm on Twitter @damrhein, or you can leave a comment right here.
If you work in a development shop similar to mine, you and many of your coworkers have more than one workstation under your desk.We use those extra machines for a variety of reasons but by and large they they tend to serve most often as foot warmers. That is not to say that they are unnecessary but rather they simply aren't used most of the time. If you try to eliminate one, you will surely need it within the next week but if your manager asks if it is really necessary you would be hard pressed to pinpoint precisely when the last time it was used for something really important. To developers, these extra machines are potential sandboxes for isolated experiments or testing scenarios. For managers, they are relatively unused capital investments that require inventory control and have depreciating value. If you are a network administrator there are certainly computers in your inventory that are older and lack the capacity to be counted on for everyday use. They sit in a corner or in a blade rack and are probably idle or even powered off. These assets take up physical space and contribute very little to your data center. However, they have little sale value but may represent a significant investment. Or maybe you just can't part with them for sentimental reasons.
Whatever the reasons for having computing resources lying around that are seldom used, here is an idea: Virtualization. With virtualized images you can use those machines for whatever purposes are required and for as long as they are required without having to spend hours loading them with a compliant OS image, installing software and configuring them for use. Virtual image libraries could hold preinstalled systems for almost any need. It could be for anything:
Workstations provisioned for temporary workers
More server capacity
More machines or load testing
Extra processors for parallel processing systems
Back up systems to carry loads during maintenance hours
If you use WebSphere in any capacity, CloudBurst can be used to lay in place a completely functioning WebSphere install in as little as 20 minutes, OS and all.
When the need for the machine is passed, it can be un-deployed and returned to the pool. This could significantly increase the available computing power of an entire development business. The ability to turn any machine into a needed and useful system on demand is real agile computing and gives a whole new dimension to governance.
When IBM Workload Deployer v3.0 rolled around, the appliance introduced the concept of shared services. These were services that a cloud administrator could launch into the cloud infrastructure defined to IBM Workload Deployer, and use to serve a number of different application deployments. There were, and continue to be, two main shared services: a proxy service and a cache service. The shared proxy service does pretty much what you may guess. It provides request routing capabilities across multiple different instances of multiple different applications, thereby providing a centralized resource that encapsulates this basic need in an application environment. You can probably also guess what the caching service does. It caches things! Specifically, in IBM Workload Deployer v3.0 it provided in-memory caching of HTTP sessions, thus ensuring high availability of data stored in those sessions.
Undoubtedly, the ability to make HTTP session data fault tolerant is extremely critical in any application environment, cloud-based environments included. However, the applicability of a shared cache service is much further reaching, and in IBM Workload Deployer v3.1, we are starting to open this service up to your applications. What does this mean to you? Quite simply it now means that you can access this cache directly from your application code. If you are familiar with WebSphere eXtreme Scale or the DataPower XC10 Caching Appliance, then you know exactly what I mean. You can use the WebSphere eXtreme Scale ObjectGrid API to insert, read, update, and delete entries that exist in the in-memory cache. The underlying cache technology is based on the same code that powers WebSphere eXtreme Scale and DataPower XC10, so you can be sure that your cache is scalable, fault tolerant, responsive, and otherwise able to meet the needs of your application.
As I hope you find to be the case with many IBM Workload Deployer capabilities, this is a superbly simple capability to leverage. When you deploy virtual application patterns based on the IBM Workload Deployer Pattern for Web Applications, the capability is simply there. The underlying runtime that is serving your application is automatically augmented with the capabilities necessary so that your applications can connect to and utilize the deployed caching service. It is also worth pointing out that you can utilize the caching capabilities provided by this shared service for applications and application infrastructure that you deploy via virtual system patterns as well. You can either choose to augment the WebSphere Application Server environment with the XC10 Feature Pack (a deploy-time option for virtual system patterns built on WebSphere Application Server Hypervisor Edition v8), or you can configure WebSphere Application Server as you always would when integrating with a WebSphere eXtreme Scale environment or a DataPower XC10 Appliance.
What's the real benefit to all of this you ask? Well, when you use the shared caching service, you get the benefits of a distributed, in-memory, extremely scalable cache without having to deal with too much setup or administration. You simply tell IBM Workload Deployer how many resources you want to dedicate to your cache, and deploy the shared service. IBM Workload Deployer takes care of the details, including scaling in and out the cache to meet the needs of the system. On top of all of this, there is also an option to configure 'Next to the Cloud' caching. If you currently own DataPower XC10 appliances, you can make those available to virtual application pattern deployments (this was already possible with virtual system patterns) by simply providing details of the location of the appliance collective in question.
Put simply, setting up, administering, and utilizing an object caching service for your applications has never been easier. Check it out and let us know what you think!
If you are reading this blog then I am pretty sure that you are interested in the agility that can be achieved by rapidly provisioning middleware systems and standing up virtual applications in a private cloud environment. However there are other aspects of agility that you should also consider. One such aspect is the ability to build applications that can be easily maintained, updated, and extended. This is where OSGi technology comes into the picture.
If you have been working with the IBM Workload Deployer (or watching some IBM Workload Deployer demos) you may have noticed a category of components in the virtual application builder called OSGi Components.
Maybe you already know all about OSGi applications and the value they bring to an enterprise. Or, perhaps you noticed this and decided that you would search for some more information on this odd acronym and just what an OSGi application is all about.
In a nutshell OSGi technology is a way to define dynamic modules for Java. It provides a standard way to encapsulate components (called bundles) with metadata that define versioned package dependencies, service dependencies, packages exported, services exported, etc... basically everything you need to know about this bundle so that it can be connected up with other bundles to support a particular solution. These bundles can then be grouped together into applications and dynamically wired to fulfill necessary dependencies at runtime. The OSGi framework provides all of the necessary capability to manage the dependencies and resolve any problems.
Those who leverage OSGi technology benefit from improved time-to-market and reduced development costs. The loose coupling provided by the OSGi framework reduces maintenance costs and facilitates the dynamic delivery of components in a running system. Of course there's a lot more to it than just that ... involving portability across different environments, achieving the appropriate level of isolation or sharing within an environment, and integrating with the many different technologies and patterns already available today. I don't think I know enough about OSGi to do it justice here. But fortunately for me (and you) there are several experts who can make it all clear.
One such expert is Graham Charters and there is a great opportunity to hear him introduce this topic and also participate in a dialogue about the concepts and what they mean for your business. Graham will be leading a Global WebSphere Community Lab Chat on Wednesday of this week (July 20th) entitled: How can OSGi make your enterprise more agile. Graham is the IBM technical lead in the OSGi Alliance Enterprise Expert Group and an active participant in the open source community implementing many of these standards. So register now for this free session and learn how OSGi can make your enterprise even more agile.
We've been talking a lot recently about Virtual Application Patterns and enhancements to this deployment model in IBM Workload Deployer v3.1. This is appropriate because virtual applications are a substantial evolution for application deployment in a private cloud. Virtual Application Patterns deliver on the promise of Platform-as-a-Service - letting you focus on the application while Workload Deployer builds the necessary platform to deploy and manage your application.
However, Virtual System Patterns are still alive and well ... and quite frankly, this is where many people begin to explore the functionality provided in Workload Deployer. For many, it is a logical first step to start recreating familiar physical environments in the private cloud and then leverage these environments to develop and test their applications. It is also a great way to build out new applications using familiar concepts, leveraging existing scripts, and take full advantage of the agility, consistency, and increased resource utilization available in a Workload Deployer managed private cloud.
You may recall that virtual system patterns are sometimes called topology patterns because they are used to define a topology middleware configuration to meet application requirements. With a virtual system pattern you define exactly the type of middleware configuration that you need for your application environment and Workload Deployer provisions exactly that configuration when the pattern is deployed to your private cloud.
To use an automotive analogy, you might compare virtual systems to building your own hot-rod from a molded frame while virtual applications are more like purchasing a complete vehicle from a dealer. When you purchase a vehicle from a dealer you receive a fully functional automobile. Sure, you can choose the color and some options – but you don't necessarily know the details of all of the components that make your vehicle functional. Just add a driver (you) and off you go! This saves you substantial time and money while freeing you from the need to be an automotive engineer. As with the production vehicle, virtual applications are optimized for a specific purpose and are extremely effective when used for that purpose. All you need to do is add your application (the driver) and run-time requirements. Virtual system patterns are like the hot-rod approach. You start with a modeled frame of sorts (hypervisor edition images) – thereby saving time and effort so you don't have a start from scratch. However you still have the responsibility and flexibility to create a very unique custom vehicle. Doing so requires more expertise and a greater time investment when compared to a production vehicle (virtual application), but you get to decide all of the details. With virtual systems you specify the exact vehicle you need for your application. This provides substantial flexibility but requires a deep knowledge of the middleware and an investment of time building necessary scripts and other elements to support your application environment.
So as I mentioned, virtual system patterns are very popular. And if you have been following recent posts about the enhancements delivered in IBM Workload Deployer v3.1 you noticed that several of the features primarily focused on virtual applications have at the same time been extended to virtual system patterns - such as the shared caching service and the new base AIX image. So we certainly consider virtual systems deployment model to be important. IBM Workload Deployer v3.1 delivered new hypervisor edition images and the IBM Image Construction and Composition Tool was bundled with Workload Deployer - primarily used for creating custom images to leverage in virtual system patterns. The IBM Image Construction tool is a substantial advancement in the ability to create your own custom base images.
To help communicate that we haven't been neglecting virtual system deployment patterns, I created a new demo to highlight this deployment model. The demo begins by providing a quick overview of the components that go into a virtual system pattern. It then shows how to clone a pattern to customize it for your own purpose, deploy it, monitor licenses, and monitor resource usage in your private cloud. Finally, it shows a quick demonstration of installing an emergency fix to a deployed virtual system instance.
I'll be showing this and other demos at IBM Pulse 2012 next week. I hope to see you there!
I was at a customer meeting the other day, and someone asked me if they could query WebSphere CloudBurst for an inventory of all of their virtual system deployments. This person was of course aware that he could go to the web console and very quickly view all of the virtual systems. What he wanted though was something that he could run to generate a report that contained all of this information. For a purpose like this, harnessing the WebSphere CloudBurst CLI is exactly the way to go.
I thought I'd write a simple CLI script that provides an example of how you could do this.
from datetime import datetime
outFile.write("WebSphere CloudBurst Virtual System Inventory\n")
outFile.write("Total virtual systems: " + str(len(cloudburst.virtualsystems)))
def writeVSDetails(outFile, virtualSystem):
outFile.write("\tVirtual system name: " + virtualSystem.name)
outFile.write("\tCreated from pattern: " + virtualSystem.pattern.name)
outFile.write("\tVirtual system status: " + virtualSystem.currentstatus_text)
created = datetime.fromtimestamp(virtualSystem.created)
outFile.write("\tVirtual system creation date: " + created.strftime("%B %d, %Y %H:%M:%S"))
outFile.write("\tTotal virtual machines: " + str(len(virtualSystem.virtualmachines)))
def writeVMDetails(outFile, virtualMachine):
outFile.write("\t\tVirtual machine name: " + virtualMachine.name)
outFile.write("\t\tVirtual machine display name: " + virtualMachine.displayname)
outFile.write("\t\tCreated from image: " + virtualMachine.virtualimage.name)
outFile.write("\t\tVirtual machine hypervisor: " + virtualMachine.hypervisor.name + " | " + virtualMachine.hypervisor.address)
outFile.write("\t\tVirtual machine IP address: " + virtualMachine.ip.ipaddress)
outFileLoc = sys.argv
outFile = open(outFileLoc, 'w')
for virtualSystem in cloudburst.virtualsystems:
for virtualMachine in virtualSystem.virtualmachines:
As a result of invoking this script using the CLI's batch mode, content is written to the file location supplied by the caller.
WebSphere CloudBurst Virtual System Inventory
Total virtual systems: 3
Virtual system name: Single server
Created from pattern: WebSphere single server
Virtual system status: Started
Virtual system creation date: January 15, 2010 16:37:20
Total virtual machines: 1
Virtual machine name: Standalone 0
Virtual machine display name: Single server cbvm-110 default
Created from image: WebSphere Application Server 220.127.116.11
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
Virtual system name: Development WAS Cluster
Created from pattern: Custom WAS Cluster - Development
Virtual system status: Started
Virtual system creation date: January 18, 2010 14:08:46
Total virtual machines: 2
Virtual machine name: DMGR 0
Virtual machine display name: Development WAS Cluster cbvm-112 dmgr
Created from image: WebSphere Application Server 18.104.22.168
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
Virtual machine name: Custom Node 1
Virtual machine display name: Development WAS Cluster cbvm-111 custom
Created from image: WebSphere Application Server 22.214.171.124
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
Virtual system name: DB2 for development use
Created from pattern: DB2
Virtual system status: Started
Virtual system creation date: January 18, 2010 14:09:58
Total virtual machines: 1
Virtual machine name: DB2 Enterprise Server 32bit Trial 0
Virtual machine display name: DB2 for development use cbvm-113
Created from image: DB2 Enterprise 126.96.36.199 32-bit Trial
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
I withheld IP addresses and host names above for obvious reasons, but if you ran the script against your environment you would see actual host name and IP address values. The script above is written once, and it can be subsequently run anytime you want an inventory of virtual systems running in your WebSphere CloudBurst cloud. There's other information available for virtual systems and virtual machines that I didn't show here, and you can retrieve it if necessary for your inventory report. In addition, I chose to print this information as regular text in a file supplied by the caller, but you might choose to generate the report in another format including XML, JSON, or anything else for that matter.
-- Dustin Amrhein
p.s. As with any sample code or script I provide here, the above is only a sample and offered as-is.
One of my favorite things to do with users or potential users of WebSphere CloudBurst is to help them understand how they can construct a custom environment using the appliance. Typically, we take one of their existing application environments and discuss the configuration steps that contribute to its makeup. From there, we map the required configuration actions to different customization capabilities in the appliance. It is one thing to talk about how you can customize every layer of your application stack with WebSphere CloudBurst, it is quite another to talk about it in the context of an existing environment. This exercise usually serves to greatly enhance a user's understanding of how to construct tailored environments with the appliance.
While I cannot take every one of you through this exercise in the context of one of your own application environments, I can propose a scenario that will help to illustrate the WebSphere CloudBurst customization process. Consider that I want to deploy a clustered WebSphere Application Server environment whose application server instances utilize WebSphere DataPower XC10 for HTTP session management. In order to deploy such an environment, I would need to do the following:
Install an OS and WAS
Install the WebSphere eXtreme Scale Client binaries - required for integration
Construct a clustered cell
Augment profiles with WebSphere eXtreme Scale profile templates
Configure the application server instances to use WebSphere DataPower XC10 for session management
So those are the steps, but how do they map to WebSphere CloudBurst? First, I know that the WebSphere Application Server Hypervisor Edition image used by WebSphere CloudBurst encapsulates the installation of the OS and WAS. I also know that WebSphere CloudBurst will automatically construct the clustered cell during the deployment process. That means I need to address the installation of client binaries, augmentation of profiles, and configuration of application server instances. In order to do this, I will use a combination of image extension and custom script packages.
To get started, I extend an existing WebSphere Application Server Hypervisor Edition image and simply install the WebSphere eXtreme Scale Client binaries. I then capture that image and store it as my own unique image in the WebSphere CloudBurst catalog. Now, you may wonder why I did not capture the profile augmentation in the custom image. Remember, you cannot change profile configuration during the extend and capture process as WebSphere CloudBurst resets the profiles as part of capturing the custom image.
My custom image encapsulates the installation of the client binaries, so now I turn to custom script packages. I need two in this case. One script package will augment a profile (either deployment manager or custom node) with the WebSphere eXtreme Scale profile template. The second script package will configure application server instances to use WebSphere DataPower XC10 for HTTP session management. Once done with these script packages, I have all the assets I need to build my target environment.
Using my custom image, I build a pattern that contains the number and kind of WebSphere Application Server nodes that I want. I use the advanced options to define a WebSphere Application Server cluster ensuring its creation happens during deployment. Next, I drag and drop the profile augmentation script onto the deployment manager and custom node parts in my pattern. Finally, I drag and drop the WebSphere DataPower XC10 configuration script onto the deployment manager. The pattern is now ready to deploy!
For those of you that are visual learners like me, this demonstration provides a nice overview of exactly what I wrote about above. Check it out and let me know what you think.
One of my favorite books from childhood is If You Give a Mouse a Cookie. Although targeted at children, the book illustrates a frequently occurring human behavior that is important for all of us understand. That behavior is the tendency for escalating expectations. The book offers this up by starting out with the simple action of giving a mouse a cookie. The mouse in turn asks for a glass of milk, various flavors of cookies, and on and on, until the mouse circles back to asking for another cookie.
Nearly all of us exhibit this same kind of behavior, and it can often produce positive results. In particular, in IT we always push for the next best thing or a slightly better outcome. Personally, I am no stranger to this behavior because I experience it from WebSphere CloudBurst users quite frequently. In these cases, it usually revolves around one particular outcome: speed of deployment.
Bar none, users of WebSphere CloudBurst are experiencing unprecedented deployment times for the environments they dispense through the appliance. The fact that we say you can deploy meaningful enterprise application environments in a matter of minutes is far beyond just marketing literature. Our users prove it everyday. However, just because they are deploying things faster than ever does not mean they are content to rest on those achievements. They want to push the envelope, and I love it.
For our users looking to achieve even speedier deployment times, I offer up one reminder and one tip. First, analyze all of your script packages to ensure you are using the right means of customization. If you have some scripts that run for considerably longer than most other script packages, you may want to at least consider applying that customization by creating a custom image. You still need to adhere to the customization principles outlined here, but you may benefit from applying the customization in an image once and avoiding the penalty for applying it during every deployment. You may also be able to break this customization out with a combination of a custom image and script packages. For instance, instead of having a script that installs and configures monitoring agents, you may install the agents in a custom image and configure them during deployment. Being selective about how and when you apply customizations can go a long way in improving your deployment times.
In addition to the reminder above, I also have a tip. Take a look at all of the script packages you use in pattern deployments and look to see if there are any that you can apply in an asynchronous manner. In other words, identify customizations that need to start, but not necessarily complete as part of the deployment process. Going back to our example of configuring monitoring agents during the deployment process, it may be important to kick off the configuration script during deployment, but is it crucial to wait on the results? Maybe not. If it is not, consider defining the executable argument in your script package in a manner that kicks off the execution and proceeds -- i.e. nohup executable command &. This approach can save deployment time in certain situations.
My advice to users of WebSphere CloudBurst: keep pushing your deployment process! Pare as many minutes off the process as you can. I hope that the tips above help in that regard, and be sure to pass along other techniques that you have found helpful.
When I first started to become aware of the cloud computing movement, I remember being intrigued but not all that aware of its possible consequences to me. After all, I was a developer not a systems administrator, so other than professional curiosity why was cloud computing all that important to me? Maybe you are a developer that can see right through my early, naive perception of cloud computing, but maybe you are a developer that, like me in the early going, doesn't quite see why cloud computing should matter to you. In the case of the latter, I've come to realize that there are several reasons why cloud computing matters to the developer. Let me try to sum up a few of those reasons for you here.
Reason #1: Developer services can be delivered via the cloudThere are many different types of services that can be realized from a cloud (public, private, or hybrid) that could have a large impact on the way developers work. As I mention in a previous post, IBM announced a Tools as a Service initiative in which IDEs are made available within a public cloud. IDEs in the cloud give developers a single development environment that can be accessed from any machine at any time. Better yet, we don't have to worry with installing and maintaining the environment. In addition to IDEs in the cloud, with the increased focus on virtualization and virtualization management that cloud is bringing, the ability to rapidly procure and instantiate runtime environments should become standard practice. This means that new ideas and new product code can be rapidly prototyped and tested. No longer should a proof of concept be delayed because it couldn't be proven in a runtime environment.
Reason #2: Cloud computing means a world of new products and offeringsAs a developer, it is a continuous battle to keep up with constantly emerging technologies, but it is imperative that we do so in order to ensure we take full advantage of available solutions. Cloud computing providers introduce a whole new world of service offerings for consumption by application developers. Cloud providers are offering new storage solutions, new database implementations, new content distribution mechanisms, new application integration capabilities, etc. As developers who may potentially be writing applications that run in the cloud, these new offerings directly affect the code we write. We need to educate ourselves about these new services, and we should understand when these solutions can be best leveraged to deliver our end product.
Reason #3: SOA becomes more importantOkay, so maybe this is not aimed squarely at the developer, but I know many times a developer wears the hat of architect as well... even if they don't know it! In a cloud computing world, the applications and services we deploy to the cloud should align and fit into our SOA. This is critical if we are to fully exploit the benefits of ubiquity offered by the cloud. Cloud computing inherently provides the ability to access services from any machine with a network connection, automatically giving the kind of service ubiquity sought by many companies. By developing these services in a SOA-compliant manner, we extend the reach by making it more readily consumable by other application components. We move beyond pure end-user applications and services, and in doing so new or increased revenue streams may be realized for the service.
These are just a few of the ways I see cloud computing currently affecting the developer's role. There are a myriad of reasons that developers should be cognizant of cloud computing, and I expect the list of reasons to boom as cloud computing continues to advance. I'd also like to hear what you think about cloud computing and the developer, so post a comment below if you would like to join the discussion.
Applications - just like humans, animals, plants, and many other things - have a life cycle. They are conceived, given birth, grow, do foolish things in youth, hopefully improve over time, have problems that need to be fixed, don't always age well .... and eventually they will die and release their assets to the next generation. Sounds kind of familiar, doesn't it?
One of the many benefits of virtual application patterns in IBM Workload Deployer and related IBM offerings is support for the complete life cycle of the application. You can manage the complete life cycle of virtual applications from a single interface that is fully integrated and well thought out - not just a series of links from one product UI to a different product UI. This eliminates the complexity of having to work with different interfaces, paradigms, metaphors, controls, labels, names, authorization, and so on - that is often the norm in many customer environments today. I think the benefits of this integration are obvious - eliminating confusion, configuration, miscommunication, interpretation, and mapping errors. Providing a truly complete solution also facilitates a common knowledge base and encourages cooperation and collaboration among teams. You can share patterns, providing consistent governance for a solution, guarantee consistency in deployments, and build upon the expertise provided by others. Having an integrated solution for design, development, deployment, configuration changes, monitoring, and problem determination ensures that time is not wasted and valuable information is not lost.
In the course of my job, I am lucky to be able to work with both enterprise users and business partners who are adopting and using the WebSphere CloudBurst Appliance. When it comes to the business partner camp, one of my absolute favorites is the Haddon Hill Group. The Haddon Hill Group is an IBM Premier Business Partner, and they have been an early adopter and vocal advocate of the WebSphere CloudBurst Appliance. They have extensive knowledge of the use of the appliance in enterprise accounts, and quite frankly, they are doing some really cool things with WebSphere CloudBurst.
Given the above, I was glad to see summarized results from their various implementations made available recently on the IBM site. The summary is fairly concise, so I encourage you to take a look at the results Haddon Hill Group is getting with WebSphere CloudBurst.
I am not going to rehash the contents of the results here, but there are a couple of things I want to call out. First off, Haddon Hill Group says that WebSphere CloudBurst can provide companies with a '100 times faster time to market' delivery experience. In a practical sense, this means reducing the amount of time to deliver WebSphere environments from 40-60 days on average to just hours. That is an eye-opening data point!
The other thing I want call out here is a quote from Phil Schaadt, President and CTO, Haddon Hill Group. I have had the pleasure of working with Phil and team, and I have heard him echo these same sentiments many times:
"The important thing about the IBM WebSphere CloudBurst Appliance is that it will dispense a WebSphere Application Server image onto your WebSphere Application Server environment or private cloud along with other products within the WebSphere stack, and that application server will be ready in a few minutes. You can do it in a clustered environment, and you can even roll out IBM WebSphere Process Server and get it right in a fully clustered environment with a database connection, in about 90 minutes. You can also easily manage all the configurations of IBM WebSphere Process Server that you need. All the steps that took up so much time and effort on the part of IT staff have been removed. The savings for companies with large WebSphere implementations can be in the millions."
It is always great to see clients putting our technology to use to produce tangible business value. Again, I encourage you to take a look at these reports. As always, I am eager to hear what you think, so leave me a comment or reach out to me on Twitter @damrhein.
One of the most powerful features of WebSphere CloudBurst is the ability to take one of the WebSphere Application Server Hypervisor Edition virtual images that are shipped with the appliance and extend it to a produce a custom virtual image. This allows users to begin creating customized environments from the bottom up, starting with the operating system. To put it in better context, let's take a look at a couple of scenarios where this feature comes in quite handy.
First off, a very common need for our customers is the ability to continually monitor their application environments. For instance, you may install Tivoli monitoring agents on all of your machines hosting WebSphere Application Server processes and configure those agents to report back to a management server. This is a great case for image extension in WebSphere CloudBurst.
In this scenario, you would start by extending an existing WebSphere Application Server Hypervisor Edition image. WebSphere CloudBurst creates a running virtual machine based off of the selected image, and you log into that virtual machine and install the Tivoli monitoring agents. Once the installation is done, you capture the virtual image back into the WebSphere CloudBurst catalog and use the new image to build a custom pattern. The last step is to include a script package on this custom pattern that, upon deployment, will configure the installed monitoring agents to report back to your desired management server.
Another use case is likely to be of interest to you if you are using WebSphere Virtual Enterprise (or something similar), and you could benefit from the same ease of provisioning for those environments that WebSphere CloudBurst brings to WebSphere Application Server environments. You can use the same customization combination above (image extension and custom scripts) to enable WebSphere CloudBurst to essentially dispense WebSphere Virtual Enterprise cells.
Again, this scenario starts off by extending a WebSphere Application Server Hypervisor Edition virtual image. Once the virtual machine for the extension is created by WebSphere CloudBurst, you log in and install the WebSphere Virtual Enterprise product. After the installation is done, you capture the image and store it in the catalog. Next, you build a custom pattern based off of this image and include script packages that, upon deployment, augment the various parts in the pattern from WebSphere Application Server profiles to WebSphere Virtual Enterprise profiles. (You may wonder why you wouldn't just create the WebSphere Virtual Enterprise profiles during the image extension process. This is because during image extension, you cannot make changes to the virtual disk that contains the WebSphere Application Server profiles. Any changes made to the profiles will be wiped out during the capture process.)
There are countless more scenarios for creating custom virtual images in WebSphere CloudBurst. To name a few, you may want to install JDBC drivers that are common to almost all of your application environments, install required anti-virus software, or just make operating system configuration changes. All of these things can be accomplished through the image extension and capture process. Look for an article coming out soon that will discuss and explain, in much greater detail than I provided here, the process of installing and configuring Tivoli monitoring agents in environments dispensed by WebSphere CloudBurst. In the meantime, if you have any questions or comments, drop us a line here or check out our forum.
Customization capabilities have been very important to the design of IBM Workload Deployer going back to the beginning with WebSphere CloudBurst. Having the ability to quickly spin up environments in a cloud really does little good if those environments are not customized according to your needs. If you look at the virtual system pattern capability, it is why we always had the notion of custom images, custom patterns, and custom scripts. We give you a strong foundation, and you tweak it here and there to create what you want.
Customization is not a concept unique to virtual system patterns. The virtual application model in IBM Workload Deployer supports many different mechanisms for you to tailor your cloud-based environments. You can start with the virtual application pattern types that we ship and use any components in those patterns to build a custom environment. The patterns you build can include your own configuration (within the set of configurable parameters) and include policies that you need for your environment. In looking at just the IBM Workload Deployer Pattern for Web Applications and the IBM Workload Deployer Pattern for Databases, there are quite a number of scenarios you can support with your cloud. However, what happens when you want to go a little further and color outside the lines of what we provide?
At some point you may have heard or read that the entire virtual application pattern model resides on a pluggable architecture. In effect, this means that everything about a virtual application pattern type, from the elements that show up when building a pattern to the management interface you interact with after deployment, is customizable. The fundamental unit of customization for a virtual application pattern type is a plugin. Plugins provide the know-how in terms of installing, configuring, integrating, and managing the application types supported by a given pattern. Plugins also provide metadata that control what users see as they build and manage these patterns. In short, plugins are the source of truth for virtual application patterns!
If you looked in IBM Workload Deployer, you would find the collection of plugins that support the virtual application pattern types shipped with the offering. While that is interesting, you should also know that you can supply your own plugins. That's right. You can develop a plugin, and load it directly into the appliance. This allows you to do two very important things. First, you can extend the virtual application pattern types that come with IBM Workload Deployer with any kind of functionality you deem important. This may be additional monitoring, integration with external systems, or any number of other extensions. Second, you can create new virtual application pattern types that support your desired workloads. You can support the workloads with the software of your choosing so long as you can supply the necessary know-how in your plugins. In either case, you contribute the plugin, and your customized components become first class members of the IBM Workload Deployer landscape.
Okay, so I admit that this is not necessarily news. We have supported user-contributed plugins since the release of IBM Workload Deployer. However, there is something new that significantly lowers the barrier to entry in the custom plugin game. Early last week, IBM announced the IBM Workload Plugin Development Kit. This kit provides a set of tools and samples designed to make the construction and packaging of custom plugins a simple process. In my opinion, this reiterates our commitment to an extensible, application-centric cloud approach, and it represents a huge step forward in the industry as a whole. Be sure to check this out, and don't be shy with the comments and feedback!