Since bundles are such a core component of the IBM Image Construction and Composition Tool, I thought it would help to take a closer, more thorough look at them than I did in my post last week (if you have not already, I suggest reading the overview post before continuing). To help us in our closer examination, we will consider an example bundle I built using the IBM Image Construction and Composition Tool. The example bundle I built encapsulates the logic to install and configure WebSphere Application Server Community Edition. Let's take this step by step.
The first part of the bundle is the General section. This section allows you to provide a name and description for the bundle, the bundle ID and version, and the products represented by the bundle.
The next section of a bundle is the Requirements section. In this section, you can define the operating system and software requirements for your bundle. In the OS section, you specify the type, distribution, and version level of the OS your bundle requires. In the software section, you can indicate that your bundle requires other bundles defined in the IBM Image Construction and Composition Tool. You do this by providing the bundle ID for required bundles.
Next, we move on to the Install section of the bundle. Two major subsections make up this section. The first subsection is the Files to Copy section. Here, you provide files, via a file upload dialog or by providing a URI, and you specify a destination directory. When you add a bundle to an image and initiate the synchronization process, the IBM Image Construction and Composition Tool will automatically copy the files you list here to the specified destination directory on the virtual machine. In the sample WebSphere Application Server Community Edition bundle, I specify a single install.sh file to copy to the virtual machine.
The second major subsection of the Install section is the Command subsection. In this section, you will specify the installation command that the IBM Image Construction and Composition Tool should automatically invoke during the synchronization process. Additionally, you can define variables that you want to make available to your installation scripts. The tool makes these available as environment variables for the process within which your script runs. In the sample bundle, I tell the Image Construction and Composition Tool to invoke the install.sh script specified above, and I define parameters that specify the location of the binaries to install, the location to install the binaries on disk, and more.
The next section in a bundle is the Configuration section. The configuration section allows you to define configuration operations that provide actions that execute for each deployment of an image containing the bundle. You can define 0 to N configuration operations in a bundle, and each configuration operation definition contains three major subsections. The first is the Files to Copy subsection. This subsection is similar to the Files to Copy subsection in the Install section. You provide files or file URIs and you provide a destination directory to which the tool will copy the file. The WebSphere Application Server Community Edition bundle contains a single configuration operation called ConfigWASCE. In the Files to Copy section, I define a single file to copy into the image's activation engine directory.
The second major subsection in the configuration operation definition is the Command subsection. Like the Command subsection in the Install section of the bundle, you specify a command to execute and optionally associate variables with the command. There is a key difference between the command definition for configuration operations as opposed to installation operations. The Image Construction and Composition Tool invokes the command you specify for installation operations exactly ONCE at image creation (synchronization) time. On the other hand, commands you specify in the configuration operation definition execute EACH time someone deploys an image containing your bundle. In the sample bundle, my ConfigWASCE.sh script will automatically execute for each deployment. The tool will package the image in such a way that ensures the automatic passing of parameters defined in the Arguments list (including num_servers, WASCE_HOME, and more) to the ConfigWASCE.sh script.
The final major subsection of a configuration operation definition is the Dependencies section. This allows you to define other services on which your configuration operation is dependent. This can include other configuration operations in the same or other bundles, and it can include general operating system services. The WebSphere Application Server Community Edition sample bundle includes a few dependencies.
The Install and Configuration sections are really the meat of your bundle, but there is more. There is a Firewall section that allows you to define port ranges and associated protocols that the IBM Image Construction and Composition Tool should ensure are open when provisioning an image containing your bundle. Currently, the tool supports firewall configuration data when building images for the IBM Cloud. The Reset section of the bundle allows you to define tasks that should execute when capturing the image back into the Image Construction and Composition Tool (after synchronziation completes). This allows you to clean up the state of the image after the install completes. Reset configuration is not currently available in the alphaWorks version of the tool. Finally, there is a License section where you can define software licenses associated with your bundle. The tool automatically adds these licenes to the constructed image's metadata, thereby allowing deployment tools to prompt the user to accept all pertinent licenses. The WebSphere Application Server Community Edition sample bundle defines a product license.
Of course, once the bundle definition is complete, you can leverage it to compose and produce an image that you can use in WebSphere CloudBurst, Tivoli Provisioning Manager, or on the IBM Cloud. In the case of the WebSphere Application Server Community Edition sample bundle, I used it to create an image that I loaded into WebSphere CloudBurst and used to build patterns.
I hope this helps to provide a better idea of what bundles are all about in the Image Construction and Composition Tool. Don't forget to take a look at the overview demo and stay tuned for more to come about this new tool!
Customization capabilities have been very important to the design of IBM Workload Deployer going back to the beginning with WebSphere CloudBurst. Having the ability to quickly spin up environments in a cloud really does little good if those environments are not customized according to your needs. If you look at the virtual system pattern capability, it is why we always had the notion of custom images, custom patterns, and custom scripts. We give you a strong foundation, and you tweak it here and there to create what you want.
Customization is not a concept unique to virtual system patterns. The virtual application model in IBM Workload Deployer supports many different mechanisms for you to tailor your cloud-based environments. You can start with the virtual application pattern types that we ship and use any components in those patterns to build a custom environment. The patterns you build can include your own configuration (within the set of configurable parameters) and include policies that you need for your environment. In looking at just the IBM Workload Deployer Pattern for Web Applications and the IBM Workload Deployer Pattern for Databases, there are quite a number of scenarios you can support with your cloud. However, what happens when you want to go a little further and color outside the lines of what we provide?
At some point you may have heard or read that the entire virtual application pattern model resides on a pluggable architecture. In effect, this means that everything about a virtual application pattern type, from the elements that show up when building a pattern to the management interface you interact with after deployment, is customizable. The fundamental unit of customization for a virtual application pattern type is a plugin. Plugins provide the know-how in terms of installing, configuring, integrating, and managing the application types supported by a given pattern. Plugins also provide metadata that control what users see as they build and manage these patterns. In short, plugins are the source of truth for virtual application patterns!
If you looked in IBM Workload Deployer, you would find the collection of plugins that support the virtual application pattern types shipped with the offering. While that is interesting, you should also know that you can supply your own plugins. That's right. You can develop a plugin, and load it directly into the appliance. This allows you to do two very important things. First, you can extend the virtual application pattern types that come with IBM Workload Deployer with any kind of functionality you deem important. This may be additional monitoring, integration with external systems, or any number of other extensions. Second, you can create new virtual application pattern types that support your desired workloads. You can support the workloads with the software of your choosing so long as you can supply the necessary know-how in your plugins. In either case, you contribute the plugin, and your customized components become first class members of the IBM Workload Deployer landscape.
Okay, so I admit that this is not necessarily news. We have supported user-contributed plugins since the release of IBM Workload Deployer. However, there is something new that significantly lowers the barrier to entry in the custom plugin game. Early last week, IBM announced the IBM Workload Plugin Development Kit. This kit provides a set of tools and samples designed to make the construction and packaging of custom plugins a simple process. In my opinion, this reiterates our commitment to an extensible, application-centric cloud approach, and it represents a huge step forward in the industry as a whole. Be sure to check this out, and don't be shy with the comments and feedback!
During the week of IMPACT this year, we announced the launch of the WebSphere CloudBurst Samples Gallery. You can go to this gallery to find and download sample script packages, CLI scripts, and other tools that we hope help you in your endeavors with the appliance. The samples are free to use and offered in an "as-is" fashion.
While I certainly will not write about each and every sample we post out there, I did want to bring your awareness to a new one I just put up today. The new sample is neither a CLI script nor a script package, though you will find it in the script packages section of the gallery. Instead, the new sample is a tool that you can run to produce WebSphere CloudBurst script packages.
Specifically, the tool runs against a target WebSphere cell to produce a WebSphere CloudBurst script package that encapsulates that cell's configuration. The tool works by running the backupConfig command against the target cell. It packages the ZIP file that results from running the command into a special WebSphere CloudBurst script package that you can include in patterns which match the source cell in node quantity and type.
The script package produced by the tool packages logic to run the restoreConfig command using the backed up configuration from the source cell. This will apply the source configuration to a new WebSphere Application Server cell created as the result of deploying a pattern. In addition, the script package contains logic to handle the possibility of changing cell, node, and host names in the target environment.
The tool’s purpose is to help you accelerate the process of importing your existing WebSphere Application Server environments into the appliance as patterns (which is a problem I believe many of you would like to solve). It certainly does not handle everything you need to do to import environments. In fact, it has the same limitations as the backupConfig/restoreConfig utilities in WebSphere Application Server. However, I do believe that it makes it a little easier to start moving your existing environments into the appliance as new WebSphere CloudBurst patterns.
Check out this video to see a quick overview of the tool, and then go download it for free from the samples gallery. The ZIP file that you download has a readme file that gives specific detail about how to use this sample tool. As always, please let me know if you have any questions or feedback.
For the next installment of this series of FAQs, let's move from product positioning and integration, square into the land of operational procedure. For this post, we will consider you are getting ready to deploy a pattern based on the WebSphere Application Server Hypervisor Edition. During the deployment process, you provide configuration information, which includes a password for a user named virtuser.
You read the documentation, and you understand that virtuser is both an operating system user and the user that WebSphere CloudBurst configures as the primary administrative user for WebSphere Application Server. Naturally, this user owns the WebSphere Application Server processes that run in the virtual machine. While it is convenient that this is all pre-configured for you, you want to know one thing: "Can I define a user besides virtuser?"
It certainly would not be the first time this question came up. The short answer to this is yes, but there are of course caveats. You can define another user and have that user own the WebSphere Application Server processes, but you cannot completely remove the virtuser user, nor should you remove virtuser as the primary administrative user. The reason for this is that WebSphere CloudBurst relies on virtuser when it carries out certain actions such as applying maintenance, applying fixes, or otherwise interacting with the WebSphere Application Server environment.
All that being said, I recently put together a script package that allows you to utilize a user other than virtuser. I hope to put the script package in our samples gallery soon, but here's a basic overview of using the script package and what it does:
Attach the script package to all parts in a pattern that contain a WebSphere Application Server process.
Deploy the pattern and provide the necessary parameter values. These include the name of the new user, a password, a common name, and a surname. The last two bits are necessary when creating a new administrative user in WebSphere Application Server.
During deployment, the script package first creates a new OS user with the specified password.
The script adds the new user to the existing OS users group.
The script creates a new WebSphere Application Server user with the same username and password and grants administrative privileges to the user.
The script shuts down the WebSphere Application Server processes.
The script changes the runAsUser value for all servers to the empty string and sets the runAsGroup value for those servers to users. This allows members of the OS users group to start the WebSphere Application Server process.
The script starts the WebSphere Application Server processes.
There are a few other activities in the script, but that should give you a basic overview. Again, note that the script does not remove the virtuser user or change that user's OS or WebSphere Application Server permissions in anyway. I would also point out that if you use WebSphere CloudBurst to apply maintenance to the WebSphere Application Server environment, it will do so as virtuser and it will restart processes as virtuser, so plan accordingly.
I hope this sheds some light on a very common question. I hope to get the sample up soon, and as always let me know if you have any questions.
Maybe you remember, but not long ago I wrote a post about scenarios when WebSphere CloudBurst and Rational Automation Framework for WebSphere (RAFW) combine to form quite the pair. You can read that post for details, but the basic scenarios were configuring and capturing, importing existing environments into WebSphere CloudBurst, and migrating from virtual to physical installations. Well, after talking with customers and colleagues lately, you can add another scenario to the list: version-to-version WebSphere Application Server migrations.
I want to be clear here about one thing before I go further. I am in no way advocating against the use of the migration tooling that ships with WebSphere Application Server. It is an excellent tool that can make migrations simple and fast. I am merely pointing out that when it comes to version-to-version migrations you have options, and you should survey them all before making a decision.
With that understanding, let's take a look at WebSphere CloudBurst and RAFW in the context of a version-to-version migration. This integrated approach to migration is ideal if you are amenable to moving up to a newer version of WebSphere Application Server in a cloud-based environment. Using both products makes migrations fast and easy, and you can be very confident that the configuration of the migrated environment is faithful to the original. The figure below shows the basic flow of the migration and breaks it down into a set of discrete steps.
Now, for a quick break down of each step:
Extract config & apps from old environment: The first step involves pointing RAFW at your existing configuration, the one you want to migrate from, and using an out-of-the-box action to import all of the configuration into a RAFW environment. You can also import your application binaries in this step.
Store config & apps from old environment: In step two, you will store the extracted configuration and application binaries in a source control repository or some backup location separate from your RAFW server. This is an optional, but recommended step.
Analyze and update apps: Before migrating your applications to the newer version of WebSphere Application Server, you can use the completely free Application Migration Toolkit to analyze the source code of your applications. This toolkit will recommend any required updates to ensure your application continues to behave as expected when moving to the new version. Again, this is an optional step, but the toolkit is free and very handy. So, why not?
Deploy new version of the environment: Step four starts by building a new WebSphere CloudBurst pattern. This new pattern matches the topology of the environment you are migrating from, and you build it from an image containing the version of WebSphere Application Server to which you want to migrate. Once built, you deploy it to your private cloud and you have a running environment in minutes.
Apply stored config and deploy updated apps: Now that you have your new environment up and running, use RAFW to apply the configuration you extracted from your old environment. RAFW inherently understands any configuration translation that needs to occur to apply the old configuration to your new environment, and it can also deploy your updated applications for you.
That's the basic overview for version-to-version migrations when you are moving to a cloud-based environment. In time, I will be posting more information about this process to shed a little more light about what is going on under the covers. In the meantime, you know how to reach me if you have questions!
I hardly ever have a conversation about WebSphere CloudBurst, or generally cloud computing for application middleware, without the topic of databases coming up. Databases are such an important piece of nearly every application middleware environment, so users want to be sure that whatever they do for their application servers, they can also do for the databases on which their applications rely. That is why the capability to deploy DB2 from WebSphere CloudBurst has been around for as nearly as long as the capability to deploy WebSphere Application Server.
Even though DB2 deployment capability has been around for a while, there are still some common misconceptions regarding the offering. First, I have talked to a fair number of users who are under the impression that we only offer a trial version of DB2 for deployment via WebSphere CloudBurst. While that was true for the first few months of the offering, that is no longer the case. For several months now, a fully supported, 64 bit, production-ready DB2 image has been ready for use in WebSphere CloudBurst. If you were waiting for a DB2 image that you could go live with, wait no longer!
The other misconception, or rather, point of confusion, arises from the fact that the DB2 image for WebSphere CloudBurst is not, by name, a Hypervisor Edition image. I can assure you that is in name only. The DB2 image looks like and behaves like any other IBM Hypervisor Edition image once you load it into the appliance. You can use it to build and deploy patterns in the same way you use other images in WebSphere CloudBurst. You may just have trouble finding it if you search for 'DB2 Hypervisor Edition' as opposed to 'DB2 Server for WebSphere CloudBurst Appliance.'
Instead of going into further detail, I want to refer you to a blog entry from a fellow IBMer, Leon Katsnelson. Leon is a program director for DB2 and is responsible for the team that develops and delivers the DB2 image for WebSphere CloudBurst. In his most recent post, he provides a nice overview of the image and gives good information for those looking to use DB2 and WebSphere CloudBurst (there is also a bit on cloud computing at the beginning that I think is spot on). Check out Leon's post, and let us know what you think!
I'm out at the RSA conference in San Francisco this week, and I'm expecting a lot of good conversations about WebSphere CloudBurst and security. This topic always comes up when I'm out and talking to customers, and I approach it from a few different angles.
First of all, WebSphere CloudBurst enables the creation of on-premise clouds (clouds in your data center). This means that you retain control over the resources that make up and support your cloud, and you have the ability to very tightly secure said resources. Notice that I say "you have the ability". I'm careful to point out that on-premise clouds do not inherently make your environment secure. If you don't already have a robust security strategy in place within your enterprise, then simply moving to a cloud model will not solve much. That being said, if you do have a comprehensive security strategy in place, one built around customized processes and access rights, then on-premise clouds are likely to make much more sense for you.
Moving beyond the opportunity for customized security controls provided by on-premise clouds, WebSphere CloudBurst delivers additional, unique security features. It starts on the outside with the tamper-resistant physical casing. If a malicious user attempts to remove the casing to get to the inner contents, the appliance is put into a dormant state, and it must be sent to IBM to be reset. "So what!" you say. If the user removes the casing and gets to the contents, couldn't they simply read the contents off the flash memory or hard disks directly, or insert them into another WebSphere CloudBurst Appliance and read them from there? Nope. All of the contents stored on the appliance's flash memory and hard disks are encrypted with a private key that cannot be changed and is unique to each and every appliance.
If you are at all familiar with WebSphere CloudBurst, you know that the appliance dispenses and monitors virtual systems running on a collection of hypervisors. Obviously then, the appliance must remotely communicate with the hypervisors. In order to secure this communication, all information between WebSphere CloudBurst and the hypervisors (and vice versa) is encrypted. This encryption is achieved by using an SSL certificate that is exchanged when a hypervisor is defined in WebSphere CloudBurst. This certificate must be accepted by a user, thus preventing rogue hypervisors from being defined in WebSphere CloudBurst.
Finally, WebSphere CloudBurst provides for the definition of users and user groups with varying permissions and resource access rights in the appliance. You don't have to turn over the keys to your cloud kingdom when you add a user to the appliance. You have the capability to define varying permissions (from simply deploying patterns, to creating them, all the way up to administering the cloud and appliance), and you have the ability to control access to resources (patterns, virtual images, script packages, cloud groups, etc.) at a fine-grained level. These two capabilities combine to allow you to control not only what actions a user can take, but also on which resources they can take those actions.
WebSphere CloudBurst was designed with focus on delivering a secure cloud experience, and I think it hit the mark. I'm sure I didn't address all your WebSphere CloudBurst and security related questions. If you have something specific in mind, leave a comment on the blog or reach out to me on Twitter. I'll do my best to address your question.
If you are going to install and use WebSphere CloudBurst in your own environment, it is very likely that you would want at least two appliances. Perhaps you want to have a standby appliance in case of a failure on the main appliance, or maybe you have different teams that are looking to utilize the appliance in different data centers. In any case, once you install multiple appliances there's another requirement that will pop up pretty quickly. Naturally you are going to want to share custom artifacts among the various WebSphere CloudBurst boxes.
When I say custom artifacts, namely I mean virtual images, patterns, and script packages. Script packages have been easy enough to share since WebSphere CloudBurst 1.0 because you can simply download the ZIP file from one appliance and upload it to another. However, there are some enhancements in WebSphere CloudBurst 1.1 that make it easy to share both patterns and images among your different appliances.
As far as patterns go, there is a new script included in the samples directory of the WebSphere CloudBurst command line interface package called patternToPython.py. This script will transform a pattern you specify into a python script. The resulting python script can then be run against a different WebSphere CloudBurst (using the CLI), and the result is the pattern is created on the target appliance. You need to be sure that the artifacts that pattern references (script packages and virtual images) exist on the target appliance and have the exact same name as they do on the appliance from which the pattern was taken. There are no other caveats, and this new sample script makes it really simple to move patterns between appliances.
For virtual images, a new feature was added that allows you to export a virtual image from the WebSphere CloudBurst console. Simply select a virtual image, specify a remote machine (any machine with SCP enabled), and click a button to export the image as an OVA file. This OVA file can then be added to another WebSphere CloudBurst catalog using the normal process for adding virtual images. You can see this feature in action here.
Stay tuned for more information about some of the handy new features in WebSphere CloudBurst 1.1. We also should have a comprehensive look at the new release coming soon in a developerWorks article.
One of my favorite books from childhood is If You Give a Mouse a Cookie. Although targeted at children, the book illustrates a frequently occurring human behavior that is important for all of us understand. That behavior is the tendency for escalating expectations. The book offers this up by starting out with the simple action of giving a mouse a cookie. The mouse in turn asks for a glass of milk, various flavors of cookies, and on and on, until the mouse circles back to asking for another cookie.
Nearly all of us exhibit this same kind of behavior, and it can often produce positive results. In particular, in IT we always push for the next best thing or a slightly better outcome. Personally, I am no stranger to this behavior because I experience it from WebSphere CloudBurst users quite frequently. In these cases, it usually revolves around one particular outcome: speed of deployment.
Bar none, users of WebSphere CloudBurst are experiencing unprecedented deployment times for the environments they dispense through the appliance. The fact that we say you can deploy meaningful enterprise application environments in a matter of minutes is far beyond just marketing literature. Our users prove it everyday. However, just because they are deploying things faster than ever does not mean they are content to rest on those achievements. They want to push the envelope, and I love it.
For our users looking to achieve even speedier deployment times, I offer up one reminder and one tip. First, analyze all of your script packages to ensure you are using the right means of customization. If you have some scripts that run for considerably longer than most other script packages, you may want to at least consider applying that customization by creating a custom image. You still need to adhere to the customization principles outlined here, but you may benefit from applying the customization in an image once and avoiding the penalty for applying it during every deployment. You may also be able to break this customization out with a combination of a custom image and script packages. For instance, instead of having a script that installs and configures monitoring agents, you may install the agents in a custom image and configure them during deployment. Being selective about how and when you apply customizations can go a long way in improving your deployment times.
In addition to the reminder above, I also have a tip. Take a look at all of the script packages you use in pattern deployments and look to see if there are any that you can apply in an asynchronous manner. In other words, identify customizations that need to start, but not necessarily complete as part of the deployment process. Going back to our example of configuring monitoring agents during the deployment process, it may be important to kick off the configuration script during deployment, but is it crucial to wait on the results? Maybe not. If it is not, consider defining the executable argument in your script package in a manner that kicks off the execution and proceeds -- i.e. nohup executable command &. This approach can save deployment time in certain situations.
My advice to users of WebSphere CloudBurst: keep pushing your deployment process! Pare as many minutes off the process as you can. I hope that the tips above help in that regard, and be sure to pass along other techniques that you have found helpful.
In a previous post, entitled Layers of Elasticity, I talked about the new dynamic virtual machine operations in WebSphere CloudBurst. Specifically, I showed you how to use the WebSphere CloudBurst web console to add more virtual machines (nodes) to an existing virtual system. Well, you can do this with the WebSphere CloudBurst command line interface as well.
First, let's assume I start off with a basic WAS ND environment represented by the pattern below:
When I deploy this pattern in WebSphere CloudBurst, I end up with two virtual machines: one for the deployment manager with an embedded IHS instance, one for my custom node federated into the cell. After deployment, suppose I want to use the CLI to interact with this virtual system. Assuming the name of my virtual system is Cluster, I can view my custom node virtual machine with the following CLI code:
The call to the clone function above takes care of creating a new profile and federating the new node into the cell. In addition, WebSphere CloudBurst automatically invokes any script packages from the source virtual machine marked to run at virtual system creation. All because of this single line of code!
The WebSphere CloudBurst CLI is a powerful interface that enables you to automate the function of the appliance. Check it out, become familiar with it, and make WebSphere CloudBurst processes a seamless part of your overall data center management approach.