IBM Impact 2012 was just last week with a theme of "Change the Game" ... and I'm still reveling in all of the excitement and energy that goes into conferences such as this. I was fortunate to get a last minute spot to attend the conference and help out at the Solution Center where I had the chance to speak to a lot of customers and other IBMers interested in cloud computing. Among the many things that stood out - there is certainly a lot of interest in cloud computing and patterns of expertise - it also seems that folks are ready to get some first hand experience with these patterns. There's plenty of opportunity for that!
One of the things that often comes up at some point in IBM Workload Deployer conversations is the notion of self-service access. Specifically, users want to know what the appliance provides that enables them to allow various teams in their organization to directly deploy the middleware environments they need. In other words, they want to use IBM Workload Deployer to tear down the traditional barriers that exist between those that request the environment and those that fulfill said request. Now, as we begin to elaborate on this notion, it becomes quickly apparent that in order to effectively enable self-service, IBM Workload Deployer must deliver a few things.
First, IBM Workload Deployer must provide the means to define users with various levels of access. Second, IBM Workload Deployer must provide the means to define resource access at a fine-grained level to different users and groups of users. Check and check. The appliance has been doing this since the beginning of WebSphere CloudBurst. Without those two things, the conversation of self-service access would end pretty quickly. However, there is a final capability that is equally important: IBM Workload Deployer must deliver a means to limit resource consumption at a fine-grained level.
In IBM Workload Deployer there are a couple of ways to achieve this. First, you could define multiple cloud groups and allow access to those groups in a way that maps directly to resource entitlements. While that may work in some situations, others call for even more granularity. You may want to allow multiple different users or groups to access a cloud group, but you may want to allow different consumption limits for each of these groups. In this situation, you can take advantage of environment profiles and a new option when defining users of IBM Workload Deployer.
Consider the case that you have a group of developers and you want to limit their consumption of memory in the cloud. First, you start by defining your development users and for each you select Environment Profile Only as the value for the Deployment Options field.
By selecting the above value for the deployment options of a user, you restrict that user to only deploying via an environment profile as opposed to general cloud group deployments. After defining all of your development users, you may choose to organize them into a user group for easier management. At that point, you can define environment profiles and determine which ones your developers should have access to using the Access granted to field of the profile.
Within the environment profile, you can define resource consumption limits for compute resource and software licenses. For instance, you can define a limit on the amount of virtual memory consumed by all deployments using the profile. It is important to note that the limit is cumulative for ALL deployments that use the profile.
Now that all of the controls are in place, consider the deployment process for one of your development users. They pick a virtual system pattern, click the deploy icon and begin to configure the pattern for deployment. In the Choose Environment section of the deployment dialog, your development user will only be able to select the Choose profile option for deployment. Further, they will only be able to deploy using the environment profiles to which they have access.
After the deployment completes, a look at the Environment limits section in the profile shows the current usage totals.
Now suppose another development user, or even the same one, comes along and attempts to deploy another virtual system pattern even though the profile limits have already been reached. The user can initiate the deployment, but they will get a near immediate failure owing to the fact that they would exceed consumption limits if the deployment were allowed to proceed.
The same kind of enforcement occurs regardless of the resource limit type. You can use this approach to limit the consumption of CPU, virtual memory, storage, or software licenses among the various different users or groups of users you define in IBM Workload Deployer. If you combine fine-grained resource consumption limits with varying permissions and fine-grained access, I think you are on the road to truly enabling self-service in the enterprise.
Script packages are an integral part of virtual system patterns in IBM Workload Deployer. By attaching script packages to your patterns, you provide customizations particular to your unique cloud-based middleware environments. Customizations provided by script packages might include installing applications, creating application resources, integrating with external enterprise systems, and much more. The bottom line is, if you are creating virtual system patterns, you will almost certainly be creating script packages.
Largely, the act of creating a script package is independent of IBM Workload Deployer. The appliance does not dictate a particular scripting language, so all you need to do is make sure you can invoke your logic in the operating system environment. Your script package may be a wsadmin script, shell script, Java program, Perl script, and on and on. After you create the actual contents of your script package, you will then load that asset into the IBM Workload Deployer catalog.
Once loaded into the catalog, you define several attributes of your script package, including the executable command, command arguments, variables, execution time, and more. The process for defining these attributes is trivial using the intuitive UI in IBM Workload Deployer, but I wanted to take a little time to remind you of a technique I recommend to all users defining script packages. You can actually package a JSON file within the script package that defines all of the script's attributes. The format of the file is simple, and I am including an example below:
The example above is one taken from a script package in our samples gallery, and it shows the basics of which you need to be aware. Notice that in the JSON file, you can provide a name, description, unzip location, executable command, command arguments, variables, and more. You only need to ensure that the name of this JSON file is cbscript.json and that you include it at the root of the script package archive. Once you have done that, you load the script package archive into the catalog, refresh the script package details, and voila -- all the attribute definitions appear!
You may ask why I recommend this since it could seem like an unnecessary step. My answer to that is that you have to define these attributes anyway, so you might as well capture it once in the file. Once you capture it once in the file, you can ensure that if the same script needs to be reloaded, or if you need to move it to another appliance, its definition will be exactly the same (and presumably correct). I use this approach for all of my work, and for all of the samples I contribute to our gallery, and it really saves me a lot of misplaced effort that can result from typos. If you are out there creating script packages, try adopting this approach. I'm pretty sure you will be happy you did!
Customers are always impressed when they learn about the simplicity, resiliency, and rapid time to value they can received from virtual applications. However, they are usually a little mystified at how virtual applications really work. After all - they have become quite accustomed to doing things the "traditional way" where they control every aspect of their applications manually. Virtual Applications represent an entirely new way of thinking. Sure, the benefits are enormous but can you really trust them? How is it doing all of this anyway?
What seems like "magic" is really a sophisticated and coordinated set of activities driven and coordinated by IBM Workload Deployer while leveraging the expertise built into the pattern type. Yes, you can trust it because experts have worked to build the system and created to it react and respond much faster than you can. When moving away from manual processes to automated processes it is always nice to get a sense of what is really happening. I think it is just human nature. We can't really place our trust in something until we have first hand experience or understand what it is really doing ... I guess it is the critic inside each one of us. Even after you've experienced the value it is still reassuring to see and understand the "how".
It is the "how does it do that?" type of question that I attempted to answer for virtual applications in a blog post I wrote on the Expert Integrated Systems blog recently. It attempts to pull the curtain aside and describe what is actually happening to support a virtual application pattern. As with my previous post - this was written for IBM PureApplication Systems but the concepts are 100% applicable to IBM Workload Deployer. I think you will find it interesting ... Continue reading ...
Lately, I have run into multiple situations where an IBM Workload Deployer user has been trying to decide exactly how they want to create their customized images for the cloud. Essentially, they have been trying to decide whether to use the native extend and capture capabilities of IBM Workload Deployer, or to pursue the use of the Image Construction and Composition Tool (also included with the appliance). The conversations have been interesting and challenging, but more importantly, they have been a reminder that constructing enterprise-ready environments for the cloud does not happen by magic. It takes thought, deliberate planning, sustainable design, and the tools to carry everything out.
The tools part we have covered. I have every confidence, bolstered by user experience after user experience, that IBM Workload Deployer and associated tools (like the Image Construction and Composition Tool) equip you to build highly customized, cloud-based application environments. In this post, I want to focus in on the thought process that goes into how you decide to build your customized environment. Specifically, I would like to talk about important points to consider as you try to understand whether to use the native extend and capture capabilities of IBM Workload Deployer or the Image Construction and Composition Tool.
To be clear from the outset, I am not trying to provide a decision flowchart in this post. For all intents and purposes, that would be next to impossible. Instead, I want to pose to you some important questions that you should ask of yourself, along with the reasons why I believe those queries to be important. Keeping in mind that this is not an all-inclusive list, here it goes:
Question: Are the customizations that you want to make congruent with an IBM-supplied image?
Reason: One of the first decisions you should make is whether or not you can start with an IBM-supplied image as the base for your customization. You need to know what middleware elements (type and version) make up your environment and what operating system should host that environment (version and distribution). You can match that information against the list of content that IBM supplies. If there is a match, you should start by looking at extend and capture to customize that image to meet your needs. If there is no direct match, you may be looking at the Image Construction and Composition Tool.
Question: Does your custom content supplement middleware content supplied in an IBM image?
Reason: If you simply need to add additional components that supplement software already in an IBM image, I believe it is best to first examine the use of extend and capture. Whether these components are IBM software or not is irrelevant as the extend and capture functionality does not care.
Question: How configurable do you want to make the custom content in your image?
Reason: If you are adding content into the image, you need to think about just how configurable you need it to be. When you use extend and capture, you add the content to an existing image in a manner that pretty well ends up being opaque to IBM Workload Deployer. To configure that content, you need to have script packages and make sure they are part of every pattern you create based on the image. Alternatively, if you use the Image Construction and Composition Tool, you can embed configuration behavior in the image's activation engine, and you can expose deploy-time parameters without needing to include script packages in every single pattern. As an example, if you need to add a monitoring agent into your environment, you would likely do this via extend and capture and end up with a pretty simple script package to configure that agent during deployment. If however, you need to create an image with a custom database, you would likely favor the Image Construction and Composition Tool as you could embed common deploy-time configuration parameters directly in the image. For a database, there are likely to be many more deploy-time configuration parameters that you want to expose as compared to a more simple monitoring agent.
Question: Is your main focus on making operating system changes?
Reason:If your primary focus is on making operating system changes AND the answer to the first question is that your target content aligns well with IBM-supplied images, then extend and capture is where you want to start. Of course, you need to make sure that you can make all necessary changes to the OS with extend and capture, but I will say that this capability is not very restrictive at all.
Admittedly, this is a short list, but I believe it is a good starting point for how you decide upon one approach versus the other. Also, I would be remiss not to point out that these tools are absolutely not mutually exclusive. Many users I work with use a combination of the two approaches. In fact, there are some use cases that call for both tools. Start by creating a completely custom image in the Image Construction and Composition Tool, and then subject that image to the extend and capture process in IBM Workload Deployer to customize it for a particular purpose, team, project, etc. I hope you find this helpful, and I welcome your feedback or thoughts!
When IBM Workload Deployer v3.0 rolled around, the appliance introduced the concept of shared services. These were services that a cloud administrator could launch into the cloud infrastructure defined to IBM Workload Deployer, and use to serve a number of different application deployments. There were, and continue to be, two main shared services: a proxy service and a cache service. The shared proxy service does pretty much what you may guess. It provides request routing capabilities across multiple different instances of multiple different applications, thereby providing a centralized resource that encapsulates this basic need in an application environment. You can probably also guess what the caching service does. It caches things! Specifically, in IBM Workload Deployer v3.0 it provided in-memory caching of HTTP sessions, thus ensuring high availability of data stored in those sessions.
Undoubtedly, the ability to make HTTP session data fault tolerant is extremely critical in any application environment, cloud-based environments included. However, the applicability of a shared cache service is much further reaching, and in IBM Workload Deployer v3.1, we are starting to open this service up to your applications. What does this mean to you? Quite simply it now means that you can access this cache directly from your application code. If you are familiar with WebSphere eXtreme Scale or the DataPower XC10 Caching Appliance, then you know exactly what I mean. You can use the WebSphere eXtreme Scale ObjectGrid API to insert, read, update, and delete entries that exist in the in-memory cache. The underlying cache technology is based on the same code that powers WebSphere eXtreme Scale and DataPower XC10, so you can be sure that your cache is scalable, fault tolerant, responsive, and otherwise able to meet the needs of your application.
As I hope you find to be the case with many IBM Workload Deployer capabilities, this is a superbly simple capability to leverage. When you deploy virtual application patterns based on the IBM Workload Deployer Pattern for Web Applications, the capability is simply there. The underlying runtime that is serving your application is automatically augmented with the capabilities necessary so that your applications can connect to and utilize the deployed caching service. It is also worth pointing out that you can utilize the caching capabilities provided by this shared service for applications and application infrastructure that you deploy via virtual system patterns as well. You can either choose to augment the WebSphere Application Server environment with the XC10 Feature Pack (a deploy-time option for virtual system patterns built on WebSphere Application Server Hypervisor Edition v8), or you can configure WebSphere Application Server as you always would when integrating with a WebSphere eXtreme Scale environment or a DataPower XC10 Appliance.
What's the real benefit to all of this you ask? Well, when you use the shared caching service, you get the benefits of a distributed, in-memory, extremely scalable cache without having to deal with too much setup or administration. You simply tell IBM Workload Deployer how many resources you want to dedicate to your cache, and deploy the shared service. IBM Workload Deployer takes care of the details, including scaling in and out the cache to meet the needs of the system. On top of all of this, there is also an option to configure 'Next to the Cloud' caching. If you currently own DataPower XC10 appliances, you can make those available to virtual application pattern deployments (this was already possible with virtual system patterns) by simply providing details of the location of the appliance collective in question.
Put simply, setting up, administering, and utilizing an object caching service for your applications has never been easier. Check it out and let us know what you think!
We've been talking a lot recently about Virtual Application Patterns and enhancements to this deployment model in IBM Workload Deployer v3.1. This is appropriate because virtual applications are a substantial evolution for application deployment in a private cloud. Virtual Application Patterns deliver on the promise of Platform-as-a-Service - letting you focus on the application while Workload Deployer builds the necessary platform to deploy and manage your application.
However, Virtual System Patterns are still alive and well ... and quite frankly, this is where many people begin to explore the functionality provided in Workload Deployer. For many, it is a logical first step to start recreating familiar physical environments in the private cloud and then leverage these environments to develop and test their applications. It is also a great way to build out new applications using familiar concepts, leveraging existing scripts, and take full advantage of the agility, consistency, and increased resource utilization available in a Workload Deployer managed private cloud.
You may recall that virtual system patterns are sometimes called topology patterns because they are used to define a topology middleware configuration to meet application requirements. With a virtual system pattern you define exactly the type of middleware configuration that you need for your application environment and Workload Deployer provisions exactly that configuration when the pattern is deployed to your private cloud.
To use an automotive analogy, you might compare virtual systems to building your own hot-rod from a molded frame while virtual applications are more like purchasing a complete vehicle from a dealer. When you purchase a vehicle from a dealer you receive a fully functional automobile. Sure, you can choose the color and some options – but you don't necessarily know the details of all of the components that make your vehicle functional. Just add a driver (you) and off you go! This saves you substantial time and money while freeing you from the need to be an automotive engineer. As with the production vehicle, virtual applications are optimized for a specific purpose and are extremely effective when used for that purpose. All you need to do is add your application (the driver) and run-time requirements. Virtual system patterns are like the hot-rod approach. You start with a modeled frame of sorts (hypervisor edition images) – thereby saving time and effort so you don't have a start from scratch. However you still have the responsibility and flexibility to create a very unique custom vehicle. Doing so requires more expertise and a greater time investment when compared to a production vehicle (virtual application), but you get to decide all of the details. With virtual systems you specify the exact vehicle you need for your application. This provides substantial flexibility but requires a deep knowledge of the middleware and an investment of time building necessary scripts and other elements to support your application environment.
So as I mentioned, virtual system patterns are very popular. And if you have been following recent posts about the enhancements delivered in IBM Workload Deployer v3.1 you noticed that several of the features primarily focused on virtual applications have at the same time been extended to virtual system patterns - such as the shared caching service and the new base AIX image. So we certainly consider virtual systems deployment model to be important. IBM Workload Deployer v3.1 delivered new hypervisor edition images and the IBM Image Construction and Composition Tool was bundled with Workload Deployer - primarily used for creating custom images to leverage in virtual system patterns. The IBM Image Construction tool is a substantial advancement in the ability to create your own custom base images.
To help communicate that we haven't been neglecting virtual system deployment patterns, I created a new demo to highlight this deployment model. The demo begins by providing a quick overview of the components that go into a virtual system pattern. It then shows how to clone a pattern to customize it for your own purpose, deploy it, monitor licenses, and monitor resource usage in your private cloud. Finally, it shows a quick demonstration of installing an emergency fix to a deployed virtual system instance.
I'll be showing this and other demos at IBM Pulse 2012 next week. I hope to see you there!
WebSphere configuration management practices are common items of conversation that comes up when I am talking with users about IBM Workload Deployer (formerly WebSphere CloudBurst). This conversation can take on so many different avenues that it is hard to capture all of them in a short blog post. So, for the sake of this post, let's consider two facets of WebSphere configuration management. The first facet is addressing the need to consistently arrive at the same configuration across multiple deployments of a given WebSphere environment. The second facet involves managing the configuration of a deployed environment over time to protect against living drift. What is the best way to tackle these two challenges? Well, it comes down to picking the right tool for the job.
When it comes to ensuring consistency of initial WebSphere configuration from deployment to deployment, there is really no better means than patterns-based deployments enabled by IBM Workload Deployer. Whether you are using a virtual system or virtual application pattern, the bottom line is that you are representing your middleware application environments as a single, directly deployable unit. When you need to stand that environment up, you simply deploy the pattern. The deployment encapsulates the installation, configuration, and integration of the environment, and your applications if you so choose. The benefit of this approach is that once you get your pattern nailed down, you can be extremely confident that the initial configuration of your environments is extremely consistent from deploy to deploy. Basically, no more bad deployments because someone forgot to run configuration step 33 out of 100!
Because we talk about the benefits of consistency provided by our IBM Workload Deployer patterns, users often ask what IBM Workload Deployer does in terms of configuration governance for deployed environments. In other words, they ask how IBM Workload Deployer helps them to track configuration changes or compare the configuration of a deployed environment to a known good one. The honest answer is that this is a bit beyond the functional domain of the appliance. While IBM Workload Deployer does allow you to manage the deployed environment (apply fixes, update deployed applications, snapshot, etc.), it does not layer some of the common configuration governance concerns on top of that. However, there is a good reason why the appliance does not focus on that. It's because Rational Automation Framework for WebSphere does!
If you find yourself wanting to actively track configuration changes, periodically (and automatically at specified intervals) compare configuration changes to a 'golden' baseline, import configurations of a known good environment, apply common configuration across a number of cells, then the capabilities of RAFW would likely be of interest to you. It can do all this and give you an incredible toolbox of out-of-the-box application deployment and configuration capabilities for WebSphere environments. In my mind, for those that spend a good deal of time dealing with WebSphere configuration, whether it be deploying applications, configuring containers, or debugging inadvertent changes, an examination of RAFW functionality is a must.
Now it is time for a bit of disclaimer/clarification. I am not suggesting that you pick one or the other when it comes to IBM Workload Deployer and RAFW. In fact, there are many scenarios where 1+1=3 with these two solutions, and I have written about it many, many times (including this article). That said, I think it is important to highlight the relative strengths of each product, so that it is easier to map it back to your pain points. In honesty, many of the users I talk with have challenges in getting the initial configuration right AND managing it over time. That kind of problem beckons for the integrated IBM Workload Deployer/RAFW solution.
Of course, technology only gets you so far when it comes to these kinds of problems. It would be disingenuous of me to suggest otherwise. It has always been and will continue to be important to establish clear and rigorous processes around the way you deploy, manage, and change environments. This just gives you an idea of some of the tools you can leverage to aid in the implementation of those processes.
Virtual Application Patterns are one of the major new features in IBM Workload Deployer v3. You've heard this concept discussed on this blog before and it is really a revolutionary way to manage your applications in a private cloud environment. With Virtual Application Patterns you provide declarative information about your application including functional and non-functional requirements of that application. You get to focus on the application rather than the middleware configuration and IBM Workload Deployer takes care of all the details necessary to launch your application with the criteria you specify. This application-centric approach radically simplifies the deployment of applications in a private cloud. And it is not just the deployment that is simplified; it is also the monitoring, metering, logging, security, caching, etc ... that is consolidated and simplified as well. Everything is custom tailored for the particular application type to provide a significant level of integration and optimization for elastic, efficient, multi-tenant, automated management and execution of that application workload.
In IBM Workload Deployer v3 there are two different types of virtual application patterns provided out of the box; a pattern for web applications and a pattern for database applications. It's no accident that these are also the two most heavily utilized types of applications in most enterprises. Of course more patterns will be appearing in the future and you have the opportunity to create your own custom patterns ... but these first two patterns can cover a substantial number of current application workloads.
So why am I introducing all of this again? Well, I want to make you aware of a new article that was just published which covers virtual application patterns in a very consumable way with enough detail and screen shots to get you started down this path. It is appropriately named: Easy virtual app automation using Workload Deployer . It really does a great job of covering not only the web application pattern - but it also introduces the database pattern (DBaaS) and shared services. If you are about to embark on virtual applications this is a great place to start.