I hardly ever have a conversation about WebSphere CloudBurst, or generally cloud computing for application middleware, without the topic of databases coming up. Databases are such an important piece of nearly every application middleware environment, so users want to be sure that whatever they do for their application servers, they can also do for the databases on which their applications rely. That is why the capability to deploy DB2 from WebSphere CloudBurst has been around for as nearly as long as the capability to deploy WebSphere Application Server.
Even though DB2 deployment capability has been around for a while, there are still some common misconceptions regarding the offering. First, I have talked to a fair number of users who are under the impression that we only offer a trial version of DB2 for deployment via WebSphere CloudBurst. While that was true for the first few months of the offering, that is no longer the case. For several months now, a fully supported, 64 bit, production-ready DB2 image has been ready for use in WebSphere CloudBurst. If you were waiting for a DB2 image that you could go live with, wait no longer!
The other misconception, or rather, point of confusion, arises from the fact that the DB2 image for WebSphere CloudBurst is not, by name, a Hypervisor Edition image. I can assure you that is in name only. The DB2 image looks like and behaves like any other IBM Hypervisor Edition image once you load it into the appliance. You can use it to build and deploy patterns in the same way you use other images in WebSphere CloudBurst. You may just have trouble finding it if you search for 'DB2 Hypervisor Edition' as opposed to 'DB2 Server for WebSphere CloudBurst Appliance.'
Instead of going into further detail, I want to refer you to a blog entry from a fellow IBMer, Leon Katsnelson. Leon is a program director for DB2 and is responsible for the team that develops and delivers the DB2 image for WebSphere CloudBurst. In his most recent post, he provides a nice overview of the image and gives good information for those looking to use DB2 and WebSphere CloudBurst (there is also a bit on cloud computing at the beginning that I think is spot on). Check out Leon's post, and let us know what you think!
In last week's post, I put the spotlight on various aspects of bundles in the Image Construction and Composition Tool. I finished with a look at a WebSphere CloudBurst virtual image created from the bundle. However, you do not just magically go from a bundle to an image that you can use in WebSphere CloudBurst, Tivoli Provisioning Manager, or on the IBM Cloud. Today, I want to show you how to go from a bundle to a custom virtual image using the IBM Image Construction and Composition Tool.
Once you have defined at least one bundle and one base operating system image, you are ready to compose a custom image. We already talked about creating a bundle, but the base operating system image is a new topic. You can do this by either starting from ISO and kickstart configuration files, or you can import an existing Open Virtual Appliance (OVA) image that contains your operating system of choice. Once you have that base image imported or defined in the Image Construction and Composition Tool, you can extend it to create a custom image on top of the base OS image.
After creating your extended image, you can add bundles that represent the software you want to install in your custom image. Simply click on the Software tab of the new virtual image. Click the add icon, and select the bundle that you want to add. You can add as many bundles as you would like to your custom image.
After adding a bundle, it will show up in the Planned list of software for the image. Click on it to display its details in the right side of the screen. You will notice General, Install, and Configuration sections for the bundle. In the Install section, you will find a list of the installation parameters you defined for the bundle. You can provide values for the parameters at this time.
If you click on the Configure section, you will see all of the configuration paramters you specified for the bundle. You can provide default values, and you can specify whether or not these should be configurable by deployers of your custom image. If you mark them as configurable, users will be able to provide values for the parameters at image deploy time, regardless of whether they provision the image using WebSphere CloudBurst, Tivoli Provisioning Manager, or the IBM Cloud.
After you add the necessary bundles and specify installation and configuration data, you can save the image. Upon saving, the image status changes from Synchronized to Out of Sync.
Now you are ready to synchronize the image. To do this, simply click the synchronize icon. This will result in the creation of a virtual machine in the cloud envrionment (VMware or IBM Cloud) you defined in the selected cloud provider. The Image Construction and Composition Tool will then invoke the appropriate installation tasks (per the bundles you included in the image) within the running virtual machine. It will also copy over any configuration scripts you defined in the bundle.
After a while, the synchronization process completes, and the image returns to the Synchronized state. At this point, you are ready to capture the image by clicking the capture icon. This results in the creation of an OVA virtual image with your customizations. When the capture process completes, the image status changes to Deployable.
Once the image is in the deployable state, it is nearly ready to use. If you are using the IBM Cloud as your cloud provider, you can simply mark the image complete by clicking the complete icon. At this point, the image will show up in your private catalog on the IBM Cloud and it is ready to use. If you are using VMware as the cloud provider, you need to export the image. Click the export icon and provide information about an SCP-enabled server to which you want to export the image. Ideally, this location is directly reachable by the WebSphere CloudBurst or Tivoli Provisioning Manager environment into which you will import the image.
You can monitor the export status in a separate window by clicking on a link shown after clicking the OK button in the dialog above. When the export finishes, you are ready to import your new custom virtual image into WebSphere CloudBurst or Tivoli Provisioning Manager.
I hope the last three posts have given you a better idea of what the new IBM Image Construction and Composition Tool is all about. There will definitely be more to come about this tool in the near future, but in the meantime, if you have any questions or comments, please reach out to me. Until then, good luck and full speed ahead on your custom image compositions!
Since bundles are such a core component of the IBM Image Construction and Composition Tool, I thought it would help to take a closer, more thorough look at them than I did in my post last week (if you have not already, I suggest reading the overview post before continuing). To help us in our closer examination, we will consider an example bundle I built using the IBM Image Construction and Composition Tool. The example bundle I built encapsulates the logic to install and configure WebSphere Application Server Community Edition. Let's take this step by step.
The first part of the bundle is the General section. This section allows you to provide a name and description for the bundle, the bundle ID and version, and the products represented by the bundle.
The next section of a bundle is the Requirements section. In this section, you can define the operating system and software requirements for your bundle. In the OS section, you specify the type, distribution, and version level of the OS your bundle requires. In the software section, you can indicate that your bundle requires other bundles defined in the IBM Image Construction and Composition Tool. You do this by providing the bundle ID for required bundles.
Next, we move on to the Install section of the bundle. Two major subsections make up this section. The first subsection is the Files to Copy section. Here, you provide files, via a file upload dialog or by providing a URI, and you specify a destination directory. When you add a bundle to an image and initiate the synchronization process, the IBM Image Construction and Composition Tool will automatically copy the files you list here to the specified destination directory on the virtual machine. In the sample WebSphere Application Server Community Edition bundle, I specify a single install.sh file to copy to the virtual machine.
The second major subsection of the Install section is the Command subsection. In this section, you will specify the installation command that the IBM Image Construction and Composition Tool should automatically invoke during the synchronization process. Additionally, you can define variables that you want to make available to your installation scripts. The tool makes these available as environment variables for the process within which your script runs. In the sample bundle, I tell the Image Construction and Composition Tool to invoke the install.sh script specified above, and I define parameters that specify the location of the binaries to install, the location to install the binaries on disk, and more.
The next section in a bundle is the Configuration section. The configuration section allows you to define configuration operations that provide actions that execute for each deployment of an image containing the bundle. You can define 0 to N configuration operations in a bundle, and each configuration operation definition contains three major subsections. The first is the Files to Copy subsection. This subsection is similar to the Files to Copy subsection in the Install section. You provide files or file URIs and you provide a destination directory to which the tool will copy the file. The WebSphere Application Server Community Edition bundle contains a single configuration operation called ConfigWASCE. In the Files to Copy section, I define a single file to copy into the image's activation engine directory.
The second major subsection in the configuration operation definition is the Command subsection. Like the Command subsection in the Install section of the bundle, you specify a command to execute and optionally associate variables with the command. There is a key difference between the command definition for configuration operations as opposed to installation operations. The Image Construction and Composition Tool invokes the command you specify for installation operations exactly ONCE at image creation (synchronization) time. On the other hand, commands you specify in the configuration operation definition execute EACH time someone deploys an image containing your bundle. In the sample bundle, my ConfigWASCE.sh script will automatically execute for each deployment. The tool will package the image in such a way that ensures the automatic passing of parameters defined in the Arguments list (including num_servers, WASCE_HOME, and more) to the ConfigWASCE.sh script.
The final major subsection of a configuration operation definition is the Dependencies section. This allows you to define other services on which your configuration operation is dependent. This can include other configuration operations in the same or other bundles, and it can include general operating system services. The WebSphere Application Server Community Edition sample bundle includes a few dependencies.
The Install and Configuration sections are really the meat of your bundle, but there is more. There is a Firewall section that allows you to define port ranges and associated protocols that the IBM Image Construction and Composition Tool should ensure are open when provisioning an image containing your bundle. Currently, the tool supports firewall configuration data when building images for the IBM Cloud. The Reset section of the bundle allows you to define tasks that should execute when capturing the image back into the Image Construction and Composition Tool (after synchronziation completes). This allows you to clean up the state of the image after the install completes. Reset configuration is not currently available in the alphaWorks version of the tool. Finally, there is a License section where you can define software licenses associated with your bundle. The tool automatically adds these licenes to the constructed image's metadata, thereby allowing deployment tools to prompt the user to accept all pertinent licenses. The WebSphere Application Server Community Edition sample bundle defines a product license.
Of course, once the bundle definition is complete, you can leverage it to compose and produce an image that you can use in WebSphere CloudBurst, Tivoli Provisioning Manager, or on the IBM Cloud. In the case of the WebSphere Application Server Community Edition sample bundle, I used it to create an image that I loaded into WebSphere CloudBurst and used to build patterns.
I hope this helps to provide a better idea of what bundles are all about in the Image Construction and Composition Tool. Don't forget to take a look at the overview demo and stay tuned for more to come about this new tool!
When I talk with WebSphere CloudBurst users, the topic of custom virtual images comes up frequently. In some cases they simply want to customize a shipped IBM Hypervisor Edition, and in other cases they want to create a completely custom image. Creating a customized version of an IBM Hypervisor Edition is relatively easy since we give you extend & capture in WebSphere CloudBurst. Creating a completely custom image has historically been a bit tougher, mostly owing ot the fact that there was not a standard tool or process for image assembly. I am happy to say that today's publication of the IBM Image Construction and Composition Tool changes all that.
Watch a demo of the IBM Image Construction and Composition Tool
The primary purpose of the Image Construction and Composition Tool is to enable a modular approach to virtual image construction, while taking into account the typical division of responsibilities within an organization. The tool allows the right people within an organization to contribute their specialized knowledge as appropriate to the virtual image creation process. This means OS teams can handle the OS and software teams can handle the appropriate software. A separate image builder can then use both OS and software components to meet the needs of users within the organization. Best of all, the image builder does not need intimate knowledge of how to install or configure any of the components in the image. They simply need to know which OS and software components to use.
When using the Image Construction and Composition Tool, you start by defining the base operating system you wish to use for your images. You can do this by importing an existing virtual image with an OS already installed, providing an ISO for the OS, or pointing to a base OS image on the IBM Cloud. The bottom line is that you have necessary flexibility to start with your certified or ‘golden’ operating system build. Once you have the base OS image defined in the Image Construction and Composition Tool, you can start defining custom software for use in the images you will compose.
In the tool, bundles represent the software you wish to install within a virtual image. The definition of a bundle contains two major parts: Installation and Configuration. The installation component of a bundle tells the Image Construction and Composition Tool how to install your software into the virtual image. You provide a script or set of scripts that install the necessary components into your image, and you direct the tool to call these scripts. These tasks run once during the initial creation of the virtual image, thus allowing you to capture large binaries, long-running installation tasks, or other necessary actions directly into your image.
The configuration section of a bundle defines actions that configure the software installed into the image. Like with the installation tasks, you provide a script or set of scripts for configuration tasks. Unlike installation tasks that run exactly once, configuration scripts become part of the image’s activation framework and as such, run during each image deployment. Using the tool, you can define input parameters for configuration scripts and optionally expose them so that users can provide values for the parameters at image deploy-time. Configuration tasks are important in providing flexibility that allows users to leverage a single virtual image for a number of different deployment scenarios.
Once you have your base OS image and one or more bundles defined in the Image Construction and Composition Tool, you can compose a virtual image. To compose a virtual image, you extend the base OS image and add any number of bundles into the new image. A base OS image plus a set of bundles defines a unique image.
After you define the image you want to construct, you initiate a synchronize action in the Image Construction and Composition Tool. When you start the synchronize action, the tool first creates a virtual machine in either a VMware or IBM Cloud environment (based on how you configured the tool). Next, the installation tasks of each bundle you included in the virtual image run to install the required software. Finally, the tool copies the configuration scripts from each bundle into the virtual machine and adds them to the image’s activation framework. This ensures the automatic invocation of all configuration scripts during subsequent image deployments.
Once the image is in the synchronized state, you can capture it. Capturing the image results in the creation of a virtual image based on the state of the synchronized virtual machine. The tool also automates the generation of metadata that becomes part of the virtual image package. When the capture of the virtual image completes, you can export it from the Image Construction and Composition Tool and deploy it using WebSphere CloudBurst, Tivoli Provisioning Manager, or the IBM Cloud.
I am excited for users to get their hands on the Image Construction and Composition Tool. I believe it represents the first big step in helping users to design and construct more sustainable virtual images. Did I mention it is completely free to download and use? Visit the Image Construction and Composition Tool website for more details and a download link. I look forward to your comments and feedback.
Though I feel like we've come a long way in some of the initial confusion surrounding IBM CloudBurst and WebSphere CloudBurst, I still get quite a few basic questions on the solutions. The two most common questions are, 'Are they different products?', and 'Can/should I use them together?'. I put together a really brief overview that answers these questions and talks about the basics of the combined solution. I hope it provides a good introduction!
One of my favorite books from childhood is If You Give a Mouse a Cookie. Although targeted at children, the book illustrates a frequently occurring human behavior that is important for all of us understand. That behavior is the tendency for escalating expectations. The book offers this up by starting out with the simple action of giving a mouse a cookie. The mouse in turn asks for a glass of milk, various flavors of cookies, and on and on, until the mouse circles back to asking for another cookie.
Nearly all of us exhibit this same kind of behavior, and it can often produce positive results. In particular, in IT we always push for the next best thing or a slightly better outcome. Personally, I am no stranger to this behavior because I experience it from WebSphere CloudBurst users quite frequently. In these cases, it usually revolves around one particular outcome: speed of deployment.
Bar none, users of WebSphere CloudBurst are experiencing unprecedented deployment times for the environments they dispense through the appliance. The fact that we say you can deploy meaningful enterprise application environments in a matter of minutes is far beyond just marketing literature. Our users prove it everyday. However, just because they are deploying things faster than ever does not mean they are content to rest on those achievements. They want to push the envelope, and I love it.
For our users looking to achieve even speedier deployment times, I offer up one reminder and one tip. First, analyze all of your script packages to ensure you are using the right means of customization. If you have some scripts that run for considerably longer than most other script packages, you may want to at least consider applying that customization by creating a custom image. You still need to adhere to the customization principles outlined here, but you may benefit from applying the customization in an image once and avoiding the penalty for applying it during every deployment. You may also be able to break this customization out with a combination of a custom image and script packages. For instance, instead of having a script that installs and configures monitoring agents, you may install the agents in a custom image and configure them during deployment. Being selective about how and when you apply customizations can go a long way in improving your deployment times.
In addition to the reminder above, I also have a tip. Take a look at all of the script packages you use in pattern deployments and look to see if there are any that you can apply in an asynchronous manner. In other words, identify customizations that need to start, but not necessarily complete as part of the deployment process. Going back to our example of configuring monitoring agents during the deployment process, it may be important to kick off the configuration script during deployment, but is it crucial to wait on the results? Maybe not. If it is not, consider defining the executable argument in your script package in a manner that kicks off the execution and proceeds -- i.e. nohup executable command &. This approach can save deployment time in certain situations.
My advice to users of WebSphere CloudBurst: keep pushing your deployment process! Pare as many minutes off the process as you can. I hope that the tips above help in that regard, and be sure to pass along other techniques that you have found helpful.
Maybe you remember, but not long ago I wrote a post about scenarios when WebSphere CloudBurst and Rational Automation Framework for WebSphere (RAFW) combine to form quite the pair. You can read that post for details, but the basic scenarios were configuring and capturing, importing existing environments into WebSphere CloudBurst, and migrating from virtual to physical installations. Well, after talking with customers and colleagues lately, you can add another scenario to the list: version-to-version WebSphere Application Server migrations.
I want to be clear here about one thing before I go further. I am in no way advocating against the use of the migration tooling that ships with WebSphere Application Server. It is an excellent tool that can make migrations simple and fast. I am merely pointing out that when it comes to version-to-version migrations you have options, and you should survey them all before making a decision.
With that understanding, let's take a look at WebSphere CloudBurst and RAFW in the context of a version-to-version migration. This integrated approach to migration is ideal if you are amenable to moving up to a newer version of WebSphere Application Server in a cloud-based environment. Using both products makes migrations fast and easy, and you can be very confident that the configuration of the migrated environment is faithful to the original. The figure below shows the basic flow of the migration and breaks it down into a set of discrete steps.
Now, for a quick break down of each step:
Extract config & apps from old environment: The first step involves pointing RAFW at your existing configuration, the one you want to migrate from, and using an out-of-the-box action to import all of the configuration into a RAFW environment. You can also import your application binaries in this step.
Store config & apps from old environment: In step two, you will store the extracted configuration and application binaries in a source control repository or some backup location separate from your RAFW server. This is an optional, but recommended step.
Analyze and update apps: Before migrating your applications to the newer version of WebSphere Application Server, you can use the completely free Application Migration Toolkit to analyze the source code of your applications. This toolkit will recommend any required updates to ensure your application continues to behave as expected when moving to the new version. Again, this is an optional step, but the toolkit is free and very handy. So, why not?
Deploy new version of the environment: Step four starts by building a new WebSphere CloudBurst pattern. This new pattern matches the topology of the environment you are migrating from, and you build it from an image containing the version of WebSphere Application Server to which you want to migrate. Once built, you deploy it to your private cloud and you have a running environment in minutes.
Apply stored config and deploy updated apps: Now that you have your new environment up and running, use RAFW to apply the configuration you extracted from your old environment. RAFW inherently understands any configuration translation that needs to occur to apply the old configuration to your new environment, and it can also deploy your updated applications for you.
That's the basic overview for version-to-version migrations when you are moving to a cloud-based environment. In time, I will be posting more information about this process to shed a little more light about what is going on under the covers. In the meantime, you know how to reach me if you have questions!
Virtual image parts play a huge role in WebSphere CloudBurst. When crafting your own customized patterns, you include anywhere from 1 to n parts from as many different virtual images as is necessary. These parts represent the different node types or personalities within a given Hypervisor Edition image, and form the basis of your pattern. When you deploy a pattern, such as the one pictured below, WebSphere CloudBurst creates a distinct virtual machine for each part.
This means that after deploying the above WebSphere Application Server pattern, you will have four virtual machines comprising your virtual system. This gives you a clean separation of concern by providing a unique container for each of your application environment nodes. This can attribute to performance optimization, increased availability, and much more. However, this approach is not suitable to all use cases. In some scenarios, especially when trying to control costs and increase consolidation, you may want to deploy a multi-node WebSphere Application Server environment within a single virtual machine. Based on what I showed you above, you might think our approach in WebSphere CloudBurst makes this impossible, but you would be overlooking an important component of patterns.
That component is of course the second building block of patterns... script packages. As you probably know, script packages allow you to supply just about any customization you want. In the case that you want a single virtual machine to host a number of WebSphere Application Server nodes, maybe even an entire cell, all you need to do is supply a script package that constructs the necessary nodes during deployment. In fact, you don't even have to write the script package. You can use the free sample in our samples gallery. As seen in the pattern below, you include this script package on a sole deployment manager part in a pattern.
The script script package provides parameters that define the node name, number of custom nodes, and number of web server nodes you want in your cell. During the deployment process, the script takes this information and constructs the cell you define. This includes creating the custom and web servers nodes and federating the custom nodes, thus completing the creation of your WebSphere Application Server cell. In this case, the script package provides deployment flexibility that is sometimes a necessity, and it is just another example of the many degrees of flexibility enabled by the script package design.
I should point out that a part in a pattern does not always map to a single node. For instance, in the case of WebSphere Process Server, there is a part that represents a complete, multi-node golden topology encapsulated within a single virtual machine. However, if you find yourself using images that do not contain these multi-node parts, rest easy knowing script packages provide you the flexibility you need.
In my last post, I concentrated on the new enhancements to WebSphere CloudBurst 18.104.22.168. One of the major new additions was the introduction of Environment Profiles, and I promised a developerWorks article would be forthcoming. The article is now live along with a demo that showcases the capability of environment profiles.
As I mentioned in my last post, environment profiles center around giving you more customization capability during the pattern deployment process. In WebSphere CloudBurst, the pattern deployment process consists of the five main steps depicted below.
Traditionally, WebSphere CloudBurst controlled the entire deployment process, thus closing it off to the deployer. Environment profiles extend the customization reach of users to be able to effect steps 1-3 in the above diagram. Specifically, environment profiles give you the following control:
Control over the assignment of IP addresses and hostnames to pattern parts: Instead of having WebSphere CloudBurst automatically assign IP addresses, and thus hostnames, to virtual machines during deployment, you can explicitly set both values during the deployment process.
Ability to deploy single patterns to multiple cloud groups: Previously, when deploying a pattern you selected a single cloud group and WebSphere CloudBurst deployed all the parts in the pattern to machines within that cloud group. While this may be okay for many cases, other cases may require you to deploy some parts of the pattern to one group of machines while other parts map to a separate set of machines. Before environment profiles, you could accomplish this with multiple patterns. With environment profiles, you can accomplish it with a single pattern.
Ability to supply virtual machine naming standards: As part of deploying a pattern, WebSphere CloudBurst creates one to many virtual machines with distinct names. Environment profiles allow you to supply a naming standard that WebSphere CloudBurst will use when creating the machines as opposed to default naming schemes previously used.
It is important to note that the use of environment profiles is completely optional, and you can continue to use the traditional deployment process, thereby leaving WebSphere CloudBurst in control. That said, the introduction of environment profiles is a direct response to consistent user feedback we received regarding the need for more control during the deployment process. Based on my user conversations, these profiles address many of said needs in an easy to use, straightforward manner. We are, of course, eager to know what you think. As always, you can let me know right here, through email, or on Twitter (@damrhein).
In keeping with the impressive release pace, WebSphere CloudBurst 22.214.171.124 is now available for download from the IBM Support site. In some ways, this release is typical of what you may expect from a service release. In other words, there are defect fixes and other general enhancements in the new version of the firmware. However, this release is a bit more than your typical service release in that there are a couple of major additions of which you should be aware.
First, starting in WebSphere CloudBurst 126.96.36.199, you will be able to provision WebSphere CloudBurst patterns to IBM Power7 systems. We already supported both Power5 and Power6 systems, and this new addition allows you to take advantage of some of the significant enhancements in the IBM Power7 hardware. In addition, this means that from a single appliance you can provision environments to multiple different releases of VMware, IBM z/VM, and IBM PowerVM hypervisor technologies. The best thing about this is that WebSphere CloudBurst provides an effective abstraction layer over the underlying infrastructure so that no matter which of the hypervisor solutions you use, the end-user experience with the appliance remains the same. You get all of this from a single device!
The other major element in WebSphere CloudBurst 188.8.131.52 is the introduction of Environment Profiles. Traditionally, WebSphere CloudBurst controlled most of the pattern deployment process. While this allows the appliance to provide a lot of configuration activity without input from the user, it made some usage scenarios hard to accommodate. For instance, historically you have not been able to assign specific IP addresses to machines in your deployment, and you have not been able to deploy a pattern across multiple cloud groups. Now, with environment profiles you can. Environment profiles give you more control over deployment behavior, including the ability to assign IP addresses (as opposed to WebSphere CloudBurst automatically doing so), deploy parts across multiple cloud groups, and apply virtual machine naming standards. The use of these new resources is completely optional, so you can still use the traditional deployment model, but this provides you with flexibility if you so choose. You can learn more about this new capability by watching the short demonstration here.
As an aside to the above information concerning WebSphere CloudBurst 184.108.40.206, I should also point out new WebSphere Hypervisor Edition announcements detailed here. I encourage you to read the announcement, but to summarize there are two major pieces of information. First, when you buy PVUs of entitlement for WebSphere Application Server Hypervisor Edition or WebSphere Message Broker Hypervisor Edition, those entitlements also apply to WebSphere Application Server and WebSphere Message Broker respectively. You can use the Hypervisor Edition images and traditional software packages in any combination, up to the level of use based on your Hypervisor Edition entitlements. Second, there is a new licensing for WebSphere Application Server Hypervisor Edition called IBM HTTP Server for WebSphere Application Server Hypervisor Edition. This allows you to deploy IBM HTTP Server instances using the WebSphere Application Server Hypervisor Edition without paying for the full Hypervisor Edition license. Again, for more details and term information, please read the announcement referenced previously.
There will be more to come about WebSphere CloudBurst 220.127.116.11, including an article on the use of environment profiles, but I wanted to give everyone a quick heads up. Let me know if you have any questions or feedback for us.
I write a lot about WebSphere CloudBurst script packages. Typically, I write about what they are, how to create them, and even provide some samples from time to time. I find that most of the time I'm either writing or talking about script packages from the standpoint that they allow you to automate the delivery of customizations to environments you deploy with WebSphere CloudBurst. More specifically, I usually explain how you can include these script packages in your patterns to ensure that your custom scripts execute as part of every pattern deployment. The truth is, that is not the whole story. In fact, it's only 1/3 of the story.
In WebSphere CloudBurst, when you define a script package you also define its execution mode. The execution mode can be one of three values, and it indicates the invocation time for the script. The default value is at virtual system creation, and that tells WebSphere CloudBurst to automatically invoke the script as part of the deployment process. This seems to be the most commonly used execution mode, and in the original version of WebSphere CloudBurst it was the only available mode (which probably attributes to why I only usually tell 1/3 of the story here). As you may expect, there is a wide range of usage scenarios for this class of script packages including installing applications, activating monitoring agents, registering cells with an externally managed DNS server, and much more.
If you are like me (and many humans), you enjoy and actually expect symmetry. In that regard, it probably comes as no surprise that there is a script package execution mode called at virtual system deletion. As the name indicates, this class of script packages executes as an automatic part of the virtual system deletion process. When a user tells WebSphere CloudBurst to remove a virtual system, before it shuts down the machines in the system, it will run each script package marked to execute at virtual system deletion. Typical use cases for these scripts include removing information about the cell from externally managed DNS servers, freeing up connections with external systems, and other external 'clean up' activities.
So this leaves the final execution mode for script packages, the when I initiate it mode to be precise. This class of script packages executes when explicitly triggered by a user. In the virtual machine detail section for a deployed virtual system, you can see a list of user-initiated script packages for a given machine. There is a start button by each of the user-initiated script packages that allows you to invoke the script when, and as many times as you need to. While these script packages have many different use cases, the most common use case is to deploy application updates. Users build these application update scripts, attach them to a pattern, and invoke them whenever they want to deliver an updated application into their already deployed environment.
WebSphere CloudBurst script packages are one of the main vehicles for delivering your customizations to your cloud environments. The three execution modes mentioned above allow you to determine when the right time to deliver those customizations is.
The concepts that govern users and user groups in WebSphere CloudBurst are fairly basic, but I get asked about them enough that I believe they warrant a short discussion. First things first, you can define users in WebSphere CloudBurst and optionally define user groups to assemble users into logical collections. For both users and user groups, you can assign roles that define the actions a particular user or group of users can take using the appliance.
All of that is straight forward, but it can get a bit tricky once we start considering the effects of user permissions when managing at the user group level. The basic premise is that when a user belongs to a group or groups, the user's effective permissions are a sum of the permissions to all of the groups to which they belong. While that is easy to say, and maybe even to understand, I feel like an example always helps.
Consider that we have a single user WCAGuy that belongs to the PatternAuthors, ContentCreators, and CloudAdmins groups. The permissions for those groups are as follows:
PatternAuthors: Users in this group have permission to create and deploy patterns
ContentCreators: Users in this group have permission to create catalog content as well as create and deploy patterns
CloudAdmins: Users in this group have permission to administer the cloud, create catalog content, and create and deploy patterns
Naturally then, it follows that the WCAGuy user can administer the cloud, create catalog content, create patterns, and deploy patterns. So then, what happens if we remove the WCAGuy user from the CloudAdmins user group? Well, as you may expect, there is an update to the user's permissions. The WCAUser user can no longer administer the cloud, but they can still create catalog content, create patterns, and deploy patterns (owing to their membership in the other two groups). Similarly, if we next removed the WCAGuy user from the ContentCreators group, then the user would retain only the permission to create and deploy patterns.
Just one more thing, let's talk about what happens when I remove a user from a group and they no longer belong to any groups. Consider that I created the WCAGuy user with the permission to create catalog content as well as create and deploy patterns. Next, I added the user to the CloudAdmins group, meaning the user now has the permission to administer the cloud. I promptly decide that the user has no business with those permissions, so I remove the user from the CloudAdmins group. What happens? The user retains the permission set of the last group to which they belonged. In this case, that means the WCAGuy user retains cloud administration rights. I have to update the user's permission set if I want to take that right away, but in this case, it will not automatically disappear upon removing them from the CloudAdmins group.
I hope this helps clear up any ambiguity you may have had concerning users, user groups, and permission sets in WebSphere CloudBurst.
Yesterday, we kicked off a WebSphere in the Clouds campaign designed to connect you with IBMers that can help you to leverage WebSphere solutions to build clouds. The campaign consists of webcasts, podcasts, live Q&A sessions, and online JAMs. You can listen to replays and sign up for upcoming events by visiting the Global WebSphere Community website.
Next week, the campaign delivers a series of podcasts that discuss the WebSphere technologies that form the building blocks of clouds. These podcasts will discuss both the business and technical aspects of these solutions, and they will cover topics like application infrastructure in the cloud, policy-based workload management using application virtualization, hybrid cloud integration, and more. Over the past few days, I had the opportunity to catch up with the various presenters of these podcasts to ask them a few questions about their solutions. These interviews provide a nice sneak peak at what is coming in the podcasts, and I will be posting them here in the coming days.
To kick things off, I'm posting a video interview with Marc Haberkorn. Marc is the WebSphere Product Manager for WebSphere CloudBurst, WebSphere Application Server Hypervisor Edition, and WebSphere Virtual Enterprise. My colleague, Ryan Boyles, caught up with Marc and got his thoughts on how these solutions enable virtualization and automation for your cloud environments. Enjoy!
I point this out about script packages because recently I put one together that is not a WebSphere Application Server administration task, but does provide configuration logic common to many WAS deployments. Specifically, I put together a script package that configures an IBM HTTP Server to be a reverse proxy server. This of course, allows clients to send requests to the IBM HTTP Server and have those requests pass through to a specified back-end destination (i.e. a service hosted on WebSphere Application Server) based on URI paths.
The script package is not all that different from many of the ones I put together. It contains a shell script (which provides configuration and orchestration logic) as well as a cbscript.json file that defines the script package's characteristics when I upload it into WebSphere CloudBurst. The notable difference in this script package is that I include a Perl script that modifies the IBM HTTP Server configuration file. This just reiterates the point that you are not limited to only wsadmin and shell scripts within your script packages.
The workings of the script package are quite straightforward. It starts with a call to the shell script that modifies the IBM HTTP Server's configuration file to ensure the loading of a couple proxy modules:
sed -i s/"#LoadModule proxy_module modules\/mod_proxy.so"/"LoadModule proxy_module modules\/mod_proxy.so"/g $HTTP_CONF
sed -i s/"#LoadModule proxy_http_module modules\/mod_proxy_http.so"/"LoadModule proxy_http_module modules\/mod_proxy_http.so"/g $HTTP_CONF
As you can see, the createProxy.pl script accepts a single argument. This argument represents the reverse proxy configuration information provided by the user during deployment. The Perl script parses the single argument and creates the appropriate proxy directives in the IBM HTTP Server's configuration file:
After the invocation of the script above, control returns to the shell script. The shell script restarts the IBM HTTP Server so that the configuration changes take place. The result is an up and running IBM HTTP Server acting as a reverse proxy based on information supplied during deployment. The listings here do not show the full script package, but I hope to have it up on our WebSphere CloudBurst Samples Gallery soon.
If the script above provides some configuration logic you can use, that is good. However, my main point for bringing it up here is to point out that WebSphere CloudBurst script packages can be more than shell and wsadmin scripts that perform WebSphere Application Server configuration tasks. You can use them to do any sort of scripted activity that is essential to your application middleware deployment process. Happy scripting!
When it comes to provisioning and managing WebSphere application environments in a cloud, nothing approaches WebSphere CloudBurst in terms of expertise and instant value. However, I bet there is more to your data center provisioning and management activities than just WebSphere application environments. You probably deploy and manage a wide variety of both IBM and non-IBM software. While some of these activities may be beyond the scope of the WebSphere expertise you get with WebSphere CloudBurst, they fall well within the reach of offerings from IBM Tivoli.
One of the Tivoli offerings that comes to mind in the service delivery automation arena is the Tivoli Service Automation Manager (TSAM). TSAM delivers capabilities to request, deploy, monitor, and manage a broad range of IT services within a cloud environment, in large part by using both virtualization and automation as delivery vehicles. Even better for WebSphere users, you can integrate TSAM and WebSphere CloudBurst to make use of TSAM capabilities in concert with the WebSphere deployment and management expertise delivered by WebSphere CloudBurst. When using these two together, you actually deploy and manage WebSphere CloudBurst patterns directly from the TSAM user interface.
The integration starts by providing information about a target WebSphere CloudBurst Appliance (essentially the location of the appliance and login credentials) within TSAM. After that, you run a discovery process included with TSAM to gather information about patterns on the target appliance. Once you discover the pattern information, you perform one last configuration step, and you are ready to go.
As far as actually initiating a pattern deployment, it works much like other project requests in TSAM. From the TSAM user interface, you create a new project based on a WebSphere CloudBurst pattern. The request goes into the queue, where an administrator can approve or reject the request. This gives a nice touch of workflow governance to WebSphere CloudBurst deployments. If approved, the project request proceeds and TSAM, by way of the WebSphere CloudBurst REST APIs, initiates the deployment of the selected pattern from the appliance. Of course, there is also a means to remove the virtual system directly from the TSAM user interface. You can cancel any WebSphere CloudBurst based project, and if approved by an administrator, TSAM again leverages the WebSphere CloudBurst REST API to trigger the deletion of the virtual system.
The integration of TSAM and WebSphere CloudBurst provides the best of both worlds really. You can use a single portal as a gateway for provisioning and managing a broad range of IT services within a cloud environment, while still leveraging the significant out-of-the-box know-how and value provided by WebSphere CloudBurst for WebSphere environments. Check out a demo of this integration here, and as always, let me know if you have any questions or comments.
In a previous post, entitled Layers of Elasticity, I talked about the new dynamic virtual machine operations in WebSphere CloudBurst. Specifically, I showed you how to use the WebSphere CloudBurst web console to add more virtual machines (nodes) to an existing virtual system. Well, you can do this with the WebSphere CloudBurst command line interface as well.
First, let's assume I start off with a basic WAS ND environment represented by the pattern below:
When I deploy this pattern in WebSphere CloudBurst, I end up with two virtual machines: one for the deployment manager with an embedded IHS instance, one for my custom node federated into the cell. After deployment, suppose I want to use the CLI to interact with this virtual system. Assuming the name of my virtual system is Cluster, I can view my custom node virtual machine with the following CLI code:
The call to the clone function above takes care of creating a new profile and federating the new node into the cell. In addition, WebSphere CloudBurst automatically invokes any script packages from the source virtual machine marked to run at virtual system creation. All because of this single line of code!
The WebSphere CloudBurst CLI is a powerful interface that enables you to automate the function of the appliance. Check it out, become familiar with it, and make WebSphere CloudBurst processes a seamless part of your overall data center management approach.
For the last post in my FAQs Revisited series, I'm going to cheat a little bit. Instead of addressing one particular question, I'm going with a grab bag of a few different questions. These are questions that I get asked quite frequently, but do not demand an entire blog post explanation. Let's get on with it.
Question: Do the new software license management capabilities provided in WebSphere CloudBurst 2.0 depend on ILMT or other supporting components?
Answer: No. The license management features are completely standalone. Of course, you can still take advantage of ILMT (through easy integration in WebSphere CloudBurst I might add) to track licenses in your cloud if you so choose.
Question: Can I deploy a pattern, make changes to my virtual system, and then recapture that as an updated pattern?
Answer: You cannot do this with WebSphere CloudBurst alone, but you can use WebSphere CloudBurst in conjunction with the Rational Automation Framework for WebSphere to do just this. Check out this article (shameless plug alert!).
Question: What if I have an urgent operating system fix to apply before IBM delivers an update to the OS in the Hypervisor Edition image?
Answer: You can either manually apply the fix to the appropriate virtual machines, or you could package up the fix as a custom WebSphere CloudBurst fix, load it into the catalog, and use the appliance to automate the application of said fix.
Question: Can I change the install location for WebSphere Application Server in the virtual image?
Answer: I've just shown you all this really cool, useful, and easy to use stuff, and you worry about install locations? Seriously though, I understand the genesis of this question usually has to do with existing scripts that assume a certain install location for WebSphere Application Server. I certainly do not advocate changing those scripts, but you cannot change the install location for WebSphere Application Server in the images. There is nothing to keep you from creating a symbolic link however.
Question: Once I deploy a pattern, what do I need to do to add more processing capacity (i.e. more application server processes)?
Answer: You have a couple of options here. You can use normal WebSphere administration techniques to add more application servers to an existing node. If that will not work (perhaps a particular node is operating at max capacity), you can use the new dynamic virtual machine operations in WebSphere CloudBurst to add an entirely new node/virtual machine. If you find yourself consistently making these types of adjustments to the runtime environment based on ebb and flow of demand, you may also want to consider the Intelligent Management Pack option for WebSphere Application Server Hypervisor Edition.
I hope this FAQs Revisited series was helpful. Stay tuned for a look at some recent work I did to integrate WebSphere CloudBurst deployments with the new WebSphere DataPower XC10 appliance.
For the next installment of this series of FAQs, let's move from product positioning and integration, square into the land of operational procedure. For this post, we will consider you are getting ready to deploy a pattern based on the WebSphere Application Server Hypervisor Edition. During the deployment process, you provide configuration information, which includes a password for a user named virtuser.
You read the documentation, and you understand that virtuser is both an operating system user and the user that WebSphere CloudBurst configures as the primary administrative user for WebSphere Application Server. Naturally, this user owns the WebSphere Application Server processes that run in the virtual machine. While it is convenient that this is all pre-configured for you, you want to know one thing: "Can I define a user besides virtuser?"
It certainly would not be the first time this question came up. The short answer to this is yes, but there are of course caveats. You can define another user and have that user own the WebSphere Application Server processes, but you cannot completely remove the virtuser user, nor should you remove virtuser as the primary administrative user. The reason for this is that WebSphere CloudBurst relies on virtuser when it carries out certain actions such as applying maintenance, applying fixes, or otherwise interacting with the WebSphere Application Server environment.
All that being said, I recently put together a script package that allows you to utilize a user other than virtuser. I hope to put the script package in our samples gallery soon, but here's a basic overview of using the script package and what it does:
Attach the script package to all parts in a pattern that contain a WebSphere Application Server process.
Deploy the pattern and provide the necessary parameter values. These include the name of the new user, a password, a common name, and a surname. The last two bits are necessary when creating a new administrative user in WebSphere Application Server.
During deployment, the script package first creates a new OS user with the specified password.
The script adds the new user to the existing OS users group.
The script creates a new WebSphere Application Server user with the same username and password and grants administrative privileges to the user.
The script shuts down the WebSphere Application Server processes.
The script changes the runAsUser value for all servers to the empty string and sets the runAsGroup value for those servers to users. This allows members of the OS users group to start the WebSphere Application Server process.
The script starts the WebSphere Application Server processes.
There are a few other activities in the script, but that should give you a basic overview. Again, note that the script does not remove the virtuser user or change that user's OS or WebSphere Application Server permissions in anyway. I would also point out that if you use WebSphere CloudBurst to apply maintenance to the WebSphere Application Server environment, it will do so as virtuser and it will restart processes as virtuser, so plan accordingly.
I hope this sheds some light on a very common question. I hope to get the sample up soon, and as always let me know if you have any questions.
When writing a new tool for the WebSphere CloudBurst samples gallery last week, I got the chance to use an API in the CLI that was new to me. Specifically, I got a chance to use the WebSphere CloudBurst CLI in order to retrieve an audit log from the appliance for a specified date period. In case this is new and interesting to you, I thought I would share what I found.
First off, let's take a look at the API I am talking about. It's pretty simple: cloudburst.audit.get(file, start, end). Here, start is the start date for the audit entries and (naturally) end is the end date for those entries. The file parameter simply denotes the location or file object you want to use to store the audit archive retrieved via the get method.
This is a simple enough API. The only wrinkle comes in dealing with calculating the start and end dates. According to the WebSphere CloudBurst Information Center, both the start and end times are 'specified as the number of seconds since midnight, January 1, 1970 UTC. Floating point values can be specified to indicate fractional seconds.' For my use case, I wanted to let a user or calling program pass the start and end times as arguments to the CLI script that retrieves the audit archive. Check out the relevant portion of my script below:
As you can see, the script takes in the start and end time in the MM/dd/yy HH:mm format (i.e. 05/20/10 15:30). It parses the value to produce a date, gets the long value of the date (which is in milliseconds according to the java.util.Date API), and divides that value by 1000. This is to account for the fact that the cloudburst.audit.get method expects you to express the start and end times in seconds. The script passes the converted dates along with the output file location to the get method. The result is a ZIP file that contains an appliance audit, license audit, and PVU audit file for the specified date range.
One of my favorite things about the WebSphere CloudBurst CLI is that it is Jython-based. This means I can leverage Java APIs from my CLI scripts, and that is huge for me because of my existing knowledge of the Java language. You certainly can substitute Python APIs for my use of Java APIs to handle the start and end date calculation. I hope this is helpful, and good luck with the WebSphere CloudBurst CLI!
Starting in WebSphere CloudBurst 2.0, there are different levels of elasticity that you can achieve in your WebSphere deployments. As I mentioned in a previous post, the Intelligent Management Pack allows you to define dynamic clusters. This means cluster membership and the number of instances of a given application adjusts on the fly to meet SLAs for your application. This enables a more dynamic environment as opposed to static cluster definitions, but there is a layer of elasticity below this that bears exploring.
Dynamic clusters work with WebSphere nodes that already exists. Users define the nodes available for use by a dynamic cluster, and the runtime uses SLAs and current system state to determine the actual nodes used and application instances started. So, what if you need more nodes than what you currently have in a given WebSphere environment? A dynamic cluster will not create a new node, so you have to define extra nodes. Starting in WebSphere CloudBurst 2.0, this is as easy as pushing a button.
Dynamic virtual machine operations allow you to add and remove nodes on the fly for a given virtual system. For instance, take the pattern in the picture below:
If you were to deploy this pattern, you would end up with a WebSphere Application Server cell with a node makeup similar to the below:
Now that the environment is out there (in mere minutes I should mention), suppose you want to add more nodes? Before WebSphere CloudBurst 2.0, you could have done it, but it would involve creating another pattern with a custom node part and deploying it. This results in two different virtual systems and complicates the maintenance stream. Now, in WebSphere CloudBurst 2.0, you can simply click a button to add a node to the existing virtual system.
From the virtual systems view, if you expand the virtual machines, by each virtual machine you will see an Actions column with a View link. If you want to add a node to the environment shown above, you simply click the View link, and then click the clone icon highlighted in green below:
WebSphere CloudBurst prompts you for the number of nodes to add. You make the selection and then click OK. The appliance creates the new node and federates it into the cell for you. For instance, if you chose to add a single node, at the end of the clone not only would you have another virtual machine in your virtual system, but also another node automatically federated into your WebSphere Application Server cell:
On the flip side, you can remove a node by clicking the delete icon in the same dialog as the clone icon above. This removes the node from the WebSphere Application Server cell and deletes the virtual machine.
The ability to easily add and remove virtual machines from your WebSphere CloudBurst virtual systems enables a very valuable level of elasticity. Now you can very easily add and remove nodes on the fly based on the current demands of your system. As always, let me know if you have any questions or comments.