Script packages are an integral part of virtual system patterns in IBM Workload Deployer. By attaching script packages to your patterns, you provide customizations particular to your unique cloud-based middleware environments. Customizations provided by script packages might include installing applications, creating application resources, integrating with external enterprise systems, and much more. The bottom line is, if you are creating virtual system patterns, you will almost certainly be creating script packages.
Largely, the act of creating a script package is independent of IBM Workload Deployer. The appliance does not dictate a particular scripting language, so all you need to do is make sure you can invoke your logic in the operating system environment. Your script package may be a wsadmin script, shell script, Java program, Perl script, and on and on. After you create the actual contents of your script package, you will then load that asset into the IBM Workload Deployer catalog.
Once loaded into the catalog, you define several attributes of your script package, including the executable command, command arguments, variables, execution time, and more. The process for defining these attributes is trivial using the intuitive UI in IBM Workload Deployer, but I wanted to take a little time to remind you of a technique I recommend to all users defining script packages. You can actually package a JSON file within the script package that defines all of the script's attributes. The format of the file is simple, and I am including an example below:
The example above is one taken from a script package in our samples gallery, and it shows the basics of which you need to be aware. Notice that in the JSON file, you can provide a name, description, unzip location, executable command, command arguments, variables, and more. You only need to ensure that the name of this JSON file is cbscript.json and that you include it at the root of the script package archive. Once you have done that, you load the script package archive into the catalog, refresh the script package details, and voila -- all the attribute definitions appear!
You may ask why I recommend this since it could seem like an unnecessary step. My answer to that is that you have to define these attributes anyway, so you might as well capture it once in the file. Once you capture it once in the file, you can ensure that if the same script needs to be reloaded, or if you need to move it to another appliance, its definition will be exactly the same (and presumably correct). I use this approach for all of my work, and for all of the samples I contribute to our gallery, and it really saves me a lot of misplaced effort that can result from typos. If you are out there creating script packages, try adopting this approach. I'm pretty sure you will be happy you did!
Virtual Application Patterns are one of the major new features in IBM Workload Deployer v3. You've heard this concept discussed on this blog before and it is really a revolutionary way to manage your applications in a private cloud environment. With Virtual Application Patterns you provide declarative information about your application including functional and non-functional requirements of that application. You get to focus on the application rather than the middleware configuration and IBM Workload Deployer takes care of all the details necessary to launch your application with the criteria you specify. This application-centric approach radically simplifies the deployment of applications in a private cloud. And it is not just the deployment that is simplified; it is also the monitoring, metering, logging, security, caching, etc ... that is consolidated and simplified as well. Everything is custom tailored for the particular application type to provide a significant level of integration and optimization for elastic, efficient, multi-tenant, automated management and execution of that application workload.
In IBM Workload Deployer v3 there are two different types of virtual application patterns provided out of the box; a pattern for web applications and a pattern for database applications. It's no accident that these are also the two most heavily utilized types of applications in most enterprises. Of course more patterns will be appearing in the future and you have the opportunity to create your own custom patterns ... but these first two patterns can cover a substantial number of current application workloads.
So why am I introducing all of this again? Well, I want to make you aware of a new article that was just published which covers virtual application patterns in a very consumable way with enough detail and screen shots to get you started down this path. It is appropriately named: Easy virtual app automation using Workload Deployer . It really does a great job of covering not only the web application pattern - but it also introduces the database pattern (DBaaS) and shared services. If you are about to embark on virtual applications this is a great place to start.
WebSphere configuration management practices are common items of conversation that comes up when I am talking with users about IBM Workload Deployer (formerly WebSphere CloudBurst). This conversation can take on so many different avenues that it is hard to capture all of them in a short blog post. So, for the sake of this post, let's consider two facets of WebSphere configuration management. The first facet is addressing the need to consistently arrive at the same configuration across multiple deployments of a given WebSphere environment. The second facet involves managing the configuration of a deployed environment over time to protect against living drift. What is the best way to tackle these two challenges? Well, it comes down to picking the right tool for the job.
When it comes to ensuring consistency of initial WebSphere configuration from deployment to deployment, there is really no better means than patterns-based deployments enabled by IBM Workload Deployer. Whether you are using a virtual system or virtual application pattern, the bottom line is that you are representing your middleware application environments as a single, directly deployable unit. When you need to stand that environment up, you simply deploy the pattern. The deployment encapsulates the installation, configuration, and integration of the environment, and your applications if you so choose. The benefit of this approach is that once you get your pattern nailed down, you can be extremely confident that the initial configuration of your environments is extremely consistent from deploy to deploy. Basically, no more bad deployments because someone forgot to run configuration step 33 out of 100!
Because we talk about the benefits of consistency provided by our IBM Workload Deployer patterns, users often ask what IBM Workload Deployer does in terms of configuration governance for deployed environments. In other words, they ask how IBM Workload Deployer helps them to track configuration changes or compare the configuration of a deployed environment to a known good one. The honest answer is that this is a bit beyond the functional domain of the appliance. While IBM Workload Deployer does allow you to manage the deployed environment (apply fixes, update deployed applications, snapshot, etc.), it does not layer some of the common configuration governance concerns on top of that. However, there is a good reason why the appliance does not focus on that. It's because Rational Automation Framework for WebSphere does!
If you find yourself wanting to actively track configuration changes, periodically (and automatically at specified intervals) compare configuration changes to a 'golden' baseline, import configurations of a known good environment, apply common configuration across a number of cells, then the capabilities of RAFW would likely be of interest to you. It can do all this and give you an incredible toolbox of out-of-the-box application deployment and configuration capabilities for WebSphere environments. In my mind, for those that spend a good deal of time dealing with WebSphere configuration, whether it be deploying applications, configuring containers, or debugging inadvertent changes, an examination of RAFW functionality is a must.
Now it is time for a bit of disclaimer/clarification. I am not suggesting that you pick one or the other when it comes to IBM Workload Deployer and RAFW. In fact, there are many scenarios where 1+1=3 with these two solutions, and I have written about it many, many times (including this article). That said, I think it is important to highlight the relative strengths of each product, so that it is easier to map it back to your pain points. In honesty, many of the users I talk with have challenges in getting the initial configuration right AND managing it over time. That kind of problem beckons for the integrated IBM Workload Deployer/RAFW solution.
Of course, technology only gets you so far when it comes to these kinds of problems. It would be disingenuous of me to suggest otherwise. It has always been and will continue to be important to establish clear and rigorous processes around the way you deploy, manage, and change environments. This just gives you an idea of some of the tools you can leverage to aid in the implementation of those processes.
When one uses IBM Workload Deployer (previously WebSphere CloudBurst) to deploy a virtual system pattern, they benefit from a completely automated deployment process. The automation includes the creation and placement of virtual machines, injection of IP addresses, initiation of internal processes, and invocation of included scripts. Most of these processes are straightforward and require little more than a brief overview. However, the placement of virtual machines stands out, and it's inner workings are the subject of quite a few questions when I discuss the appliance. With that in mind, I thought I would provide a little more information on how the placement algorithm in IBM Workload Deployer works.
The placement subsystem in IBM Workload Deployer considers three primary elements: compute resource, availability, and license optimization. Compute resource availability is the gating factor for placement. That means that IBM Workload Deployer first looks at the available CPU, memory, and storage resource in the collection of hypervisors making up the cloud group(s) you are targeting for deployment. If a particular hypervisor cannot provide enough resource based on the amount you requested for your deployment, then it is automatically taken out of the eligible hosts pool. It is important to note that IBM Workload Deployer will overcommit CPU, and it will overcommit storage if you direct it to do so. It will not overcommit memory because that could severely degrade the performance of the application(s) running in the virtual machines.
After choosing the pool of hypervisors that are capable of hosting the virtual machines in your deployment from a compute resource perspective, the appliance then considers high availability. To better understand this particular placement stage, let's consider an example. Consider you are deploying a pattern based on WebSphere Application Server Hypervisor Edition and it contains two custom node parts. It is conceivable, and in fact likely, that these two custom node parts will host members of the same cluster, and thus the two nodes will support the same applications. As such, IBM Workload Deployer will attempt to place the two custom nodes on different physical machines in order to prevent a single point of failure. Of course, this depends on having two hypervisors with enough resource (CPU, memory, storage) to host the virtual machines, but the appliance makes that decision in the first placement stage.
After considering compute resource and high availability, IBM Workload Deployer moves to the last stage of placement: license optimization. In this stage, the placement subsystem attempts to place the virtual machines on hypervisors in a way that minimizes the licensing cost to you. The appliance can do this because it is aware of IBM virtualization licensing rules and takes those into account during this stage (if you aren't familiar with virtualization licensing rules and you are curious, ask you're sales representative to explain some time). During this stage, it will not violate any resource overcommit directives or rules in place, nor will it compromise system availability, but it will seek to minimize costs within these parameters.
At this point, I should make something clear that may already have occurred to you. You can override most of these placement rules by creating a cloud group containing only one hypervisor. In this case, IBM Workload Deployer will put all virtual machines on the single hypervisor until it runs out of compute resource (memory is likely to be the constraining factor). I would not suggest that you do this unless you have a good reason or you are in a simple pilot phase, but I do like to point out the art of the possible!
While not incredibly deep from a technical perspective, I do hope that this provided a few helpful details on what goes on during the placement stages of deployment. If you have any questions, do let me know.
Though I feel like we've come a long way in some of the initial confusion surrounding IBM CloudBurst and WebSphere CloudBurst, I still get quite a few basic questions on the solutions. The two most common questions are, 'Are they different products?', and 'Can/should I use them together?'. I put together a really brief overview that answers these questions and talks about the basics of the combined solution. I hope it provides a good introduction!
One of my favorite books from childhood is If You Give a Mouse a Cookie. Although targeted at children, the book illustrates a frequently occurring human behavior that is important for all of us understand. That behavior is the tendency for escalating expectations. The book offers this up by starting out with the simple action of giving a mouse a cookie. The mouse in turn asks for a glass of milk, various flavors of cookies, and on and on, until the mouse circles back to asking for another cookie.
Nearly all of us exhibit this same kind of behavior, and it can often produce positive results. In particular, in IT we always push for the next best thing or a slightly better outcome. Personally, I am no stranger to this behavior because I experience it from WebSphere CloudBurst users quite frequently. In these cases, it usually revolves around one particular outcome: speed of deployment.
Bar none, users of WebSphere CloudBurst are experiencing unprecedented deployment times for the environments they dispense through the appliance. The fact that we say you can deploy meaningful enterprise application environments in a matter of minutes is far beyond just marketing literature. Our users prove it everyday. However, just because they are deploying things faster than ever does not mean they are content to rest on those achievements. They want to push the envelope, and I love it.
For our users looking to achieve even speedier deployment times, I offer up one reminder and one tip. First, analyze all of your script packages to ensure you are using the right means of customization. If you have some scripts that run for considerably longer than most other script packages, you may want to at least consider applying that customization by creating a custom image. You still need to adhere to the customization principles outlined here, but you may benefit from applying the customization in an image once and avoiding the penalty for applying it during every deployment. You may also be able to break this customization out with a combination of a custom image and script packages. For instance, instead of having a script that installs and configures monitoring agents, you may install the agents in a custom image and configure them during deployment. Being selective about how and when you apply customizations can go a long way in improving your deployment times.
In addition to the reminder above, I also have a tip. Take a look at all of the script packages you use in pattern deployments and look to see if there are any that you can apply in an asynchronous manner. In other words, identify customizations that need to start, but not necessarily complete as part of the deployment process. Going back to our example of configuring monitoring agents during the deployment process, it may be important to kick off the configuration script during deployment, but is it crucial to wait on the results? Maybe not. If it is not, consider defining the executable argument in your script package in a manner that kicks off the execution and proceeds -- i.e. nohup executable command &. This approach can save deployment time in certain situations.
My advice to users of WebSphere CloudBurst: keep pushing your deployment process! Pare as many minutes off the process as you can. I hope that the tips above help in that regard, and be sure to pass along other techniques that you have found helpful.
I spent most of my time growing up doing two things, going to school and playing sports. I made many fond memories -- mostly from the latter :) -- and learned more than a few lessons over that time. Of all of those lessons, there was one in particular that stuck out in both the classroom and on the baseball diamond: Sometimes you have to get back to the basics.
In that vein, I think it is time to revisit the basics of WebSphere CloudBurst. In revisiting the basics, I am not talking about the technical basics of the appliance. Rather, I am talking about revisiting exactly why WebSphere CloudBurst exists in the first place. In other words, let's take a look at the problem domains WebSphere CloudBurst addresses, and let's discuss a little bit about how the appliance does so.
A couple of weeks ago, I wrote about a sample I was working on that would allow one to apply a layer of governance to their WebSphere CloudBurst patterns. Earlier this morning, I posted the sample to the WebSphere CloudBurst Samples Gallery under the 'Sample CLI Scripts for WebSphere CloudBurst' section. The name of the new sample is 'Check WebSphere CloudBurst patterns', and you can download it here.
As hinted in my earlier post, the new sample is a simple way to check your patterns against assertions you supply in a properties file. It allows you to check that patterns contain the correct parts and scripts, and it allows you to verify that they were built from valid images. The assertion format is pretty basic, but it should be flexible enough to allow you to check patterns against a wide array of requirements. The sample archive includes a readme file that explains exactly how to use the script, and it contains a sample assertions file to give you an idea of the input syntax.
I hope this helps to address some of the requirements of many WebSphere CloudBurst users that told me they were in need of a way to apply governance to their patterns. If you have any questions about the sample, please let me know. Alternatively, if you have another idea or a problem you would like to see addressed by a sample in our gallery, please let me know.
When it comes to administration of WebSphere environments, I (and many others) am a big fan of scripting. In my view, any administrative action you carry out with frequency > 1 is ideally suited for a script. The downside to not using scripts (longer configuration times, inconsistent configurations, isolated expertise) is simply too steep in most cases. I also realize that simply saying that you should script is not enough. For some, the learning curve can be a bit daunting. Quite frequently, I talk about our samples gallery or provide posts with embedded scripts in the hopes that I can help flatten out this curve a bit.
While these samples can certainly help to speed up your scripting efforts for certain use cases, they are more or less helpful for solving tactical challenges when scripting. If you and your company are embarking down a strategic path that includes beefing up your administrative scripting capability, I would strongly suggest you look at a resource a few of my colleagues pointed me at recently.
The resource I am talking about is the wsadminlib.py package referenced here. This python script file is a collection of hundreds of methods that carry out common WebSphere Application Server administrative tasks. The authors carefully constructed these methods with clear method and parameter names. The result is a script resource that can become the foundation for your custom-crafted administrative scripts.
I recently downloaded the wsadminlib.py script and began constructing WebSphere CloudBurst script packages to utilize it. To say I am impressed would be an understatement. This file makes so many tasks so incredibly simple. Take for instance the creation of an SIBus. That's just a simple call like the following:
wsadminlib.createSIBus(clusterName, nodeName, serverName, SIBusName, scope, secure)
How about associating a shared library with an application or application module? Another one-line call:
wsadminlib.associateSharedLibrary (libName, appName, warName)
Or what about setting a custom property in the webcontainer? You guessed it. One-line:
wsadminlib.setWebContainerCustomProperty(nodeName, serverName, propName, propValue
This is just an extremely small sample of what the wsadminlib.py includes. As I mentioned earlier, there are hundreds of other methods that carry out various tasks including: installing applications, creating core groups, creating virtual hosts, installing BLAs, creating JMS queues, and much more. If you are looking to beef up your WebSphere Application Server scripting efforts, or if you are just starting, I strongly encourage you to look into and make use of this valuable resource!
Virtual image parts play a huge role in WebSphere CloudBurst. When crafting your own customized patterns, you include anywhere from 1 to n parts from as many different virtual images as is necessary. These parts represent the different node types or personalities within a given Hypervisor Edition image, and form the basis of your pattern. When you deploy a pattern, such as the one pictured below, WebSphere CloudBurst creates a distinct virtual machine for each part.
This means that after deploying the above WebSphere Application Server pattern, you will have four virtual machines comprising your virtual system. This gives you a clean separation of concern by providing a unique container for each of your application environment nodes. This can attribute to performance optimization, increased availability, and much more. However, this approach is not suitable to all use cases. In some scenarios, especially when trying to control costs and increase consolidation, you may want to deploy a multi-node WebSphere Application Server environment within a single virtual machine. Based on what I showed you above, you might think our approach in WebSphere CloudBurst makes this impossible, but you would be overlooking an important component of patterns.
That component is of course the second building block of patterns... script packages. As you probably know, script packages allow you to supply just about any customization you want. In the case that you want a single virtual machine to host a number of WebSphere Application Server nodes, maybe even an entire cell, all you need to do is supply a script package that constructs the necessary nodes during deployment. In fact, you don't even have to write the script package. You can use the free sample in our samples gallery. As seen in the pattern below, you include this script package on a sole deployment manager part in a pattern.
The script script package provides parameters that define the node name, number of custom nodes, and number of web server nodes you want in your cell. During the deployment process, the script takes this information and constructs the cell you define. This includes creating the custom and web servers nodes and federating the custom nodes, thus completing the creation of your WebSphere Application Server cell. In this case, the script package provides deployment flexibility that is sometimes a necessity, and it is just another example of the many degrees of flexibility enabled by the script package design.
I should point out that a part in a pattern does not always map to a single node. For instance, in the case of WebSphere Process Server, there is a part that represents a complete, multi-node golden topology encapsulated within a single virtual machine. However, if you find yourself using images that do not contain these multi-node parts, rest easy knowing script packages provide you the flexibility you need.