Usually when I am discussing WebSphere CloudBurst with clients, the subject of tracking usage comes up. While 'tracking usage' is pretty broad and could apply to any number of things, we often come back to two major concepts. First, users want to be able to track compute resource usage in the WebSphere CloudBurst cloud, as this helps in cloud capacity planning. Second, users want to be able to track usage by individual WebSphere CloudBurst users in order to facilitate chargeback. In both cases, WebSphere CloudBurst provides reports that help you.
When it comes to tracking compute resource usage in your WebSphere CloudBurst cloud, the appliance provides a set of pre-defined reports on the Cloud --> Machine Activity page.
As you can see from the snapshot above, WebSphere CloudBurst provides usage reports for both memory and CPU attributed to either individual hypervisors or virtual machines. In addition, the appliance tracks storage usage by device and IP usage in your cloud. For each report type, you can specify a desired date range and let WebSphere CloudBurst produce a graph showing usage over that time. The below picture shows the report for memory usage by hypervisor over a one month period.
Tracking compute resource usage is certainly important, but if your interests are mostly about using WebSphere CloudBurst to facilitate chargeback, you likely want to know about our user reports. You can find these reports on the Cloud --> User Activity page of the appliance. On this page, you will find a table that lists each user and their usage of virtual machines, CPUs, memory, and storage over a period of time that you specify. Further, you can download a comma separated value file that contains this data for further parsing or processing on your part. The image below shows an example of the user activity table.
In addition to the user usage data provided above, many WebSphere CloudBurst users find that they want to track the amount of time users had running virtual systems deployed through WebSphere CloudBurst. While the appliance does not provide a direct report with this information, you can use this free sample to calculate virtual system duration times. This free tool uses data available in the WebSphere CloudBurst audit log (data you can process to produce any custom report you need), and it calculates virtual system duration time as well as virtual system time attributed to each user. You use the WebSphere CloudBurst CLI to invoke this tool, providing it with your start and end dates for the calculation (you can find further invocation instructions inside the ZIP file containing the tool). The image below shows example output for both the virtual system duration and user virtual system time reports.
'Tracking usage' means many different things to different people and use cases. I hope the above information regarding usage tracking in WebSphere CloudBurst gives you a good idea of what you get out of the box, as well as what you can do by using the audit log (in a similar fashion to the free tool mentioned above). If you have any questions, requests, or feedback, please let me know.
Virtual image parts play a huge role in WebSphere CloudBurst. When crafting your own customized patterns, you include anywhere from 1 to n parts from as many different virtual images as is necessary. These parts represent the different node types or personalities within a given Hypervisor Edition image, and form the basis of your pattern. When you deploy a pattern, such as the one pictured below, WebSphere CloudBurst creates a distinct virtual machine for each part.
This means that after deploying the above WebSphere Application Server pattern, you will have four virtual machines comprising your virtual system. This gives you a clean separation of concern by providing a unique container for each of your application environment nodes. This can attribute to performance optimization, increased availability, and much more. However, this approach is not suitable to all use cases. In some scenarios, especially when trying to control costs and increase consolidation, you may want to deploy a multi-node WebSphere Application Server environment within a single virtual machine. Based on what I showed you above, you might think our approach in WebSphere CloudBurst makes this impossible, but you would be overlooking an important component of patterns.
That component is of course the second building block of patterns... script packages. As you probably know, script packages allow you to supply just about any customization you want. In the case that you want a single virtual machine to host a number of WebSphere Application Server nodes, maybe even an entire cell, all you need to do is supply a script package that constructs the necessary nodes during deployment. In fact, you don't even have to write the script package. You can use the free sample in our samples gallery. As seen in the pattern below, you include this script package on a sole deployment manager part in a pattern.
The script script package provides parameters that define the node name, number of custom nodes, and number of web server nodes you want in your cell. During the deployment process, the script takes this information and constructs the cell you define. This includes creating the custom and web servers nodes and federating the custom nodes, thus completing the creation of your WebSphere Application Server cell. In this case, the script package provides deployment flexibility that is sometimes a necessity, and it is just another example of the many degrees of flexibility enabled by the script package design.
I should point out that a part in a pattern does not always map to a single node. For instance, in the case of WebSphere Process Server, there is a part that represents a complete, multi-node golden topology encapsulated within a single virtual machine. However, if you find yourself using images that do not contain these multi-node parts, rest easy knowing script packages provide you the flexibility you need.
In my last post, I concentrated on the new enhancements to WebSphere CloudBurst 184.108.40.206. One of the major new additions was the introduction of Environment Profiles, and I promised a developerWorks article would be forthcoming. The article is now live along with a demo that showcases the capability of environment profiles.
As I mentioned in my last post, environment profiles center around giving you more customization capability during the pattern deployment process. In WebSphere CloudBurst, the pattern deployment process consists of the five main steps depicted below.
Traditionally, WebSphere CloudBurst controlled the entire deployment process, thus closing it off to the deployer. Environment profiles extend the customization reach of users to be able to effect steps 1-3 in the above diagram. Specifically, environment profiles give you the following control:
Control over the assignment of IP addresses and hostnames to pattern parts: Instead of having WebSphere CloudBurst automatically assign IP addresses, and thus hostnames, to virtual machines during deployment, you can explicitly set both values during the deployment process.
Ability to deploy single patterns to multiple cloud groups: Previously, when deploying a pattern you selected a single cloud group and WebSphere CloudBurst deployed all the parts in the pattern to machines within that cloud group. While this may be okay for many cases, other cases may require you to deploy some parts of the pattern to one group of machines while other parts map to a separate set of machines. Before environment profiles, you could accomplish this with multiple patterns. With environment profiles, you can accomplish it with a single pattern.
Ability to supply virtual machine naming standards: As part of deploying a pattern, WebSphere CloudBurst creates one to many virtual machines with distinct names. Environment profiles allow you to supply a naming standard that WebSphere CloudBurst will use when creating the machines as opposed to default naming schemes previously used.
It is important to note that the use of environment profiles is completely optional, and you can continue to use the traditional deployment process, thereby leaving WebSphere CloudBurst in control. That said, the introduction of environment profiles is a direct response to consistent user feedback we received regarding the need for more control during the deployment process. Based on my user conversations, these profiles address many of said needs in an easy to use, straightforward manner. We are, of course, eager to know what you think. As always, you can let me know right here, through email, or on Twitter (@damrhein).
In keeping with the impressive release pace, WebSphere CloudBurst 220.127.116.11 is now available for download from the IBM Support site. In some ways, this release is typical of what you may expect from a service release. In other words, there are defect fixes and other general enhancements in the new version of the firmware. However, this release is a bit more than your typical service release in that there are a couple of major additions of which you should be aware.
First, starting in WebSphere CloudBurst 18.104.22.168, you will be able to provision WebSphere CloudBurst patterns to IBM Power7 systems. We already supported both Power5 and Power6 systems, and this new addition allows you to take advantage of some of the significant enhancements in the IBM Power7 hardware. In addition, this means that from a single appliance you can provision environments to multiple different releases of VMware, IBM z/VM, and IBM PowerVM hypervisor technologies. The best thing about this is that WebSphere CloudBurst provides an effective abstraction layer over the underlying infrastructure so that no matter which of the hypervisor solutions you use, the end-user experience with the appliance remains the same. You get all of this from a single device!
The other major element in WebSphere CloudBurst 22.214.171.124 is the introduction of Environment Profiles. Traditionally, WebSphere CloudBurst controlled most of the pattern deployment process. While this allows the appliance to provide a lot of configuration activity without input from the user, it made some usage scenarios hard to accommodate. For instance, historically you have not been able to assign specific IP addresses to machines in your deployment, and you have not been able to deploy a pattern across multiple cloud groups. Now, with environment profiles you can. Environment profiles give you more control over deployment behavior, including the ability to assign IP addresses (as opposed to WebSphere CloudBurst automatically doing so), deploy parts across multiple cloud groups, and apply virtual machine naming standards. The use of these new resources is completely optional, so you can still use the traditional deployment model, but this provides you with flexibility if you so choose. You can learn more about this new capability by watching the short demonstration here.
As an aside to the above information concerning WebSphere CloudBurst 126.96.36.199, I should also point out new WebSphere Hypervisor Edition announcements detailed here. I encourage you to read the announcement, but to summarize there are two major pieces of information. First, when you buy PVUs of entitlement for WebSphere Application Server Hypervisor Edition or WebSphere Message Broker Hypervisor Edition, those entitlements also apply to WebSphere Application Server and WebSphere Message Broker respectively. You can use the Hypervisor Edition images and traditional software packages in any combination, up to the level of use based on your Hypervisor Edition entitlements. Second, there is a new licensing for WebSphere Application Server Hypervisor Edition called IBM HTTP Server for WebSphere Application Server Hypervisor Edition. This allows you to deploy IBM HTTP Server instances using the WebSphere Application Server Hypervisor Edition without paying for the full Hypervisor Edition license. Again, for more details and term information, please read the announcement referenced previously.
There will be more to come about WebSphere CloudBurst 188.8.131.52, including an article on the use of environment profiles, but I wanted to give everyone a quick heads up. Let me know if you have any questions or feedback for us.
I write a lot about WebSphere CloudBurst script packages. Typically, I write about what they are, how to create them, and even provide some samples from time to time. I find that most of the time I'm either writing or talking about script packages from the standpoint that they allow you to automate the delivery of customizations to environments you deploy with WebSphere CloudBurst. More specifically, I usually explain how you can include these script packages in your patterns to ensure that your custom scripts execute as part of every pattern deployment. The truth is, that is not the whole story. In fact, it's only 1/3 of the story.
In WebSphere CloudBurst, when you define a script package you also define its execution mode. The execution mode can be one of three values, and it indicates the invocation time for the script. The default value is at virtual system creation, and that tells WebSphere CloudBurst to automatically invoke the script as part of the deployment process. This seems to be the most commonly used execution mode, and in the original version of WebSphere CloudBurst it was the only available mode (which probably attributes to why I only usually tell 1/3 of the story here). As you may expect, there is a wide range of usage scenarios for this class of script packages including installing applications, activating monitoring agents, registering cells with an externally managed DNS server, and much more.
If you are like me (and many humans), you enjoy and actually expect symmetry. In that regard, it probably comes as no surprise that there is a script package execution mode called at virtual system deletion. As the name indicates, this class of script packages executes as an automatic part of the virtual system deletion process. When a user tells WebSphere CloudBurst to remove a virtual system, before it shuts down the machines in the system, it will run each script package marked to execute at virtual system deletion. Typical use cases for these scripts include removing information about the cell from externally managed DNS servers, freeing up connections with external systems, and other external 'clean up' activities.
So this leaves the final execution mode for script packages, the when I initiate it mode to be precise. This class of script packages executes when explicitly triggered by a user. In the virtual machine detail section for a deployed virtual system, you can see a list of user-initiated script packages for a given machine. There is a start button by each of the user-initiated script packages that allows you to invoke the script when, and as many times as you need to. While these script packages have many different use cases, the most common use case is to deploy application updates. Users build these application update scripts, attach them to a pattern, and invoke them whenever they want to deliver an updated application into their already deployed environment.
WebSphere CloudBurst script packages are one of the main vehicles for delivering your customizations to your cloud environments. The three execution modes mentioned above allow you to determine when the right time to deliver those customizations is.
It's really hard to complain about my work week right now. As I write this blog, I'm sitting in the Congress Center in Düsseldorf, Germany looking out over the Rhine River. As an aside, in Germany it is the Rhein River, and I have a historical connection to this body of water. My surname, Amrhein, translates (loosely) to 'on the Rhein'. It does not take an expert in genealogy to conclude that I have ancestors who at one time or another lived very close to this important German waterway.
Okay, putting the family tree aside for a minute, there is a good reason that I am in Düsseldorf this week. The city, and specifically the Congress Center, is playing host to the IBM European WebSphere Technical Conference. I am here presenting sessions that include a WebSphere CloudBurst overview, a WebSphere CloudBurst hands-on lab, and an up-close look at one of our internal team's use of the appliance. I have done each of these sessions once so far, and attendance was great, audience participation high, and feedback forthcoming. I am hearing and seeing the same thing in other sessions, which is of course, ideal for us presenters.
Now, to focus in on WebSphere CloudBurst for a bit, it seems that I am hearing a recurring question this week from the mostly European audience: "Why is WebSphere CloudBurst delivered as an appliance?" I am sure that I addressed this question in a previous blog post, but I believe it bears revisiting. There are various reasons I could give for the appliance form factor, but I like to distill all of that down into three major reasons: Consumability, Performance, and Security.
From a solution consumability perspective, nothing beats the appliance approach. WebSphere CloudBurst is an integrated hardware and software solution that delivers a specific set of function. You do not have to install software, procure and maintain storage for resources on the appliance (images, patterns, scripts, etc.), and maintain software components over time. You simply drop the appliance in to your data center, perform a one-time initialization, hook it up to the network, and you are ready to start leveraging WebSphere CloudBurst to build out your private cloud. While there is definitely work to setup the cloud infrastructure that WebSphere CloudBurst deploys environments to, we can completely eliminate a significant portion of solution implementation lead time by delivering everything you need in the appliance.
The performance benefits of an appliance approach are a natural result of building an integrated hardware and software stack. Design and development teams provide optimizations in both the hardware and software based on the fact that both the hardware and software have intimate knowledge of each other's design. In other words, this is not a 'least common denominator' tuning approach. Rather, the integrated design leads to enhanced performance for the specific set of functionality provided by WebSphere CloudBurst.
Finally, appliances enable us to deliver a very hardened, secure device. We provide private key encryption of every resource stored on the appliance. That private key is unique to each appliance and cannot be modified. In addition, the physical casing is tamper-resistant. If someone removes the casing, a 'Get Smart' style kill switch puts the appliance in a dormant state. You must send the appliance to IBM so we can reset it before further use, thus providing an additional layer of physical protection on top of the encryption. These security features, plus more, like a shield that prevents anyone from executing code on the appliance, come right out of the box and require no end-user configuration activity. In this way, you can simply focus on leveraging the user security and access controls provided by WebSphere CloudBurst.
If you had any questions on the rationale behind the appliance form factor of WebSphere CloudBurst, I hope this helps. I am off for now... back to the conference and the wonderful city of Düsseldorf.
It seems like it was announcement day across IBM, and specifically in WebSphere. While the announcements were numerous and touched many different topics, I want to focus on a couple of announcements of particular interest to those of you interested in WebSphere CloudBurst and IBM Hypervisor Edition virtual images.
First, for all of our WebSphere Process Server and WebSphere Business Monitor users, there are a couple of important pieces of information in this announcement. This announcement outlines the availability of WebSphere Business Monitor Hypervisor Edition. The new image allows you to dispense WebSphere Business Monitor 7.0 environments using WebSphere CloudBurst to VMware hypervisors. In addition, the announcement outlines the expansion of the existing WebSphere Process Server Hypervisor Edition image to support the z/VM platform and the Red Hat Enterprise Linux (RHEL) operating system for VMware.
Moving beyond our BPM set of solutions, IBM also announced the availability of a WebSphere Message Broker Hypervisor Edition. This virtual image allows you to construct and deploy WebSphere Message Broker and WebSphere MQ environments using WebSphere CloudBurst. The stack includes the RHEL operating system, and it is ready to run on VMware hypervisors.
With that in mind, here's an update to the WebSphere CloudBurst supported product matrix:
* Availability subject to dates documented in referenced announcement letters
As you can see, we are continuing our effort to expand the choice you have when using WebSphere CloudBurst to create and deploy application environments to your cloud. If you are interested in using WebSphere CloudBurst for WebSphere Business Monitor, WebSphere Process Server, or WebSphere Message Broker, check out the above announcements. You will find more technical information as well as planned availability dates.
Just one last scrap of food for thought. Feedback from you, our users, is instrumental as we continue to expand software choice with WebSphere CloudBurst. Please continue to let us know your thoughts and needs!
The concepts that govern users and user groups in WebSphere CloudBurst are fairly basic, but I get asked about them enough that I believe they warrant a short discussion. First things first, you can define users in WebSphere CloudBurst and optionally define user groups to assemble users into logical collections. For both users and user groups, you can assign roles that define the actions a particular user or group of users can take using the appliance.
All of that is straight forward, but it can get a bit tricky once we start considering the effects of user permissions when managing at the user group level. The basic premise is that when a user belongs to a group or groups, the user's effective permissions are a sum of the permissions to all of the groups to which they belong. While that is easy to say, and maybe even to understand, I feel like an example always helps.
Consider that we have a single user WCAGuy that belongs to the PatternAuthors, ContentCreators, and CloudAdmins groups. The permissions for those groups are as follows:
PatternAuthors: Users in this group have permission to create and deploy patterns
ContentCreators: Users in this group have permission to create catalog content as well as create and deploy patterns
CloudAdmins: Users in this group have permission to administer the cloud, create catalog content, and create and deploy patterns
Naturally then, it follows that the WCAGuy user can administer the cloud, create catalog content, create patterns, and deploy patterns. So then, what happens if we remove the WCAGuy user from the CloudAdmins user group? Well, as you may expect, there is an update to the user's permissions. The WCAUser user can no longer administer the cloud, but they can still create catalog content, create patterns, and deploy patterns (owing to their membership in the other two groups). Similarly, if we next removed the WCAGuy user from the ContentCreators group, then the user would retain only the permission to create and deploy patterns.
Just one more thing, let's talk about what happens when I remove a user from a group and they no longer belong to any groups. Consider that I created the WCAGuy user with the permission to create catalog content as well as create and deploy patterns. Next, I added the user to the CloudAdmins group, meaning the user now has the permission to administer the cloud. I promptly decide that the user has no business with those permissions, so I remove the user from the CloudAdmins group. What happens? The user retains the permission set of the last group to which they belonged. In this case, that means the WCAGuy user retains cloud administration rights. I have to update the user's permission set if I want to take that right away, but in this case, it will not automatically disappear upon removing them from the CloudAdmins group.
I hope this helps clear up any ambiguity you may have had concerning users, user groups, and permission sets in WebSphere CloudBurst.
I point this out about script packages because recently I put one together that is not a WebSphere Application Server administration task, but does provide configuration logic common to many WAS deployments. Specifically, I put together a script package that configures an IBM HTTP Server to be a reverse proxy server. This of course, allows clients to send requests to the IBM HTTP Server and have those requests pass through to a specified back-end destination (i.e. a service hosted on WebSphere Application Server) based on URI paths.
The script package is not all that different from many of the ones I put together. It contains a shell script (which provides configuration and orchestration logic) as well as a cbscript.json file that defines the script package's characteristics when I upload it into WebSphere CloudBurst. The notable difference in this script package is that I include a Perl script that modifies the IBM HTTP Server configuration file. This just reiterates the point that you are not limited to only wsadmin and shell scripts within your script packages.
The workings of the script package are quite straightforward. It starts with a call to the shell script that modifies the IBM HTTP Server's configuration file to ensure the loading of a couple proxy modules:
sed -i s/"#LoadModule proxy_module modules\/mod_proxy.so"/"LoadModule proxy_module modules\/mod_proxy.so"/g $HTTP_CONF
sed -i s/"#LoadModule proxy_http_module modules\/mod_proxy_http.so"/"LoadModule proxy_http_module modules\/mod_proxy_http.so"/g $HTTP_CONF
As you can see, the createProxy.pl script accepts a single argument. This argument represents the reverse proxy configuration information provided by the user during deployment. The Perl script parses the single argument and creates the appropriate proxy directives in the IBM HTTP Server's configuration file:
After the invocation of the script above, control returns to the shell script. The shell script restarts the IBM HTTP Server so that the configuration changes take place. The result is an up and running IBM HTTP Server acting as a reverse proxy based on information supplied during deployment. The listings here do not show the full script package, but I hope to have it up on our WebSphere CloudBurst Samples Gallery soon.
If the script above provides some configuration logic you can use, that is good. However, my main point for bringing it up here is to point out that WebSphere CloudBurst script packages can be more than shell and wsadmin scripts that perform WebSphere Application Server configuration tasks. You can use them to do any sort of scripted activity that is essential to your application middleware deployment process. Happy scripting!
When it comes to building and using WebSphere CloudBurst patterns, people always ask me if I have any best practices. It turns out, I do. In fact, I have a singular piece of advice that wraps it all up: Build WebSphere CloudBurst patterns in a way such that once deployed, there is no after-the-fact, manual configuration for the running environment. That means, build the pattern so that it not only contains all the nodes necessary for your application environment, but it also contains all the configuration necessary for the environment.
Put like this, most everyone I talk to agrees with me. However, they quickly recognize that, absent this really cool integration with Rational Automation Framework for WebSphere, this means they will be writing scripts for many configuration actions and including them in patterns in the form of script packages. For users not familiar with configuration scripting for our WebSphere products, this can be a daunting proposition. But... it shouldn't be!
Recently, I put together a short presentation that lays out an iterative approach for developing script packages for WebSphere CloudBurst. Specifically, the presentation focuses on developing configuration script packages for the WebSphere Application Server (though the general concepts apply to all Hypervisor Edition products equally). I believe this method is useful for anyone, from novice users to WebSphere scripting gurus. The basic process goes something like this:
Identify: Identify the target WebSphere Application Server topology and configuration for your application environment.
Deploy: Build a WebSphere CloudBurst pattern that matches your desired topology and deploy it to your cloud.
Develop and Test: Develop and test your configuration script. Not a WebSphere Application Server scripting ninja? No worries. Use the Command Assistance feature in the WebSphere Application Server v7 administration console. This feature shows you the wsadmin commands that match the actions you manually take in the console. This affords a lower barrier of entry for those not familiar with wsadmin.
Package: Package up the resulting scripts into a script package along with metadata that describes the package.
Modify and redeploy: Load the new script package into your appliance, add it to your pattern, and then redeploy. Upon deployment completion, verify the scripts produce the desired result.
The presentation provides detail on the above steps and walks through an example scenario for this process. I am embedding it below, and I hope it proves useful. As always, feel free to send in any questions or comments.