When many people think of cloud computing they immediately think of virtualization and virtual machines in particular. This is completely natural and not at all surprising. After all, one of the core underlying technologies necessary for cloud computing is virtualization. However, it is important not to confuse one element of cloud computing with the entire thing - and this can sometimes happen. Many people have begun to leverage virtual machines in their on premise environment and sometimes begin to call this their private cloud. While virtualization is a substantial step forward and help gets you started down the necessary path of standardization and automation that is essential in a cloud - it is not in and of itself "a cloud".
The National Institute of Standards and Technology has published its definition of cloud computing. This is a very complete and yet concise definition that includes not only the essential characteristics of a cloud solution but also the service models (IaaS, PaaS, SaaS) and deployment models (public, private, hybrid, community). It is a great way to get a perspective on cloud and can be useful when considering the solutions of various vendors.
Let me summarize the essential elements of cloud from this definition here:
broad network access
So, this is interesting. Not only is this much more than just virtualization - but virtualization isn't even mentioned in the list explicitly. Not to worry - virtualization is of course important and is included under the resource pooling topic. I would assert that virtualization is also necessary to facilitate the type of on-demand, self-service, elastically scaling resources that are leveraged in a cloud. What is crystal clear from this definition is that there is a lot more to a cloud solution than just virtual images and some hypervisor infrastructure upon which to run them. Somebody must provide the necessary on-demand/self-service capabilities, the network access to these services, the management of the resource pools, enabling true elasticity for running systems, measuring services and so forth. IBM Workload Deployer provides just such capabilities for the on-premise cloud allowing you to efficiently deploy patterns built for virtual systems and virtual applications with deep knowledge of the middleware that is being provisioned to optimize these solutions. Furthermore, Workload Deployer provides the complete lifecycle management from pattern creation, to deployment and provisioning, applying maintenance, resource and license management in the on-premise cloud, elastic scalability, and eventually returning resources to the on-premise cloud to be reused. Workload Deployer is a complete solution for not only server virtualization but of course for cloud computing.
However, virtualization doesn't have to stop with just virtual machines. It is a general principle that can be applied to more than just servers. At its core, virtualization is really about providing a level of abstraction between some real resources and the consumers of those resources. This is a natural fit when we think of server virtualization and virtual machines. However, there are also substantial benefits to be gained by adopting a similar abstraction between the middleware and the applications themselves - sometimes referred to as application virtualization.
By application virtualization I mean providing the capabilities to abstract the application from the underlying infrastructure such that it can be elastic, participate in health management policies, and provide agility across the pool of application infrastructure resources. This type of application virtualization is built into our Virtual Application pattern (hence the name) in Workload Deployer and surfaced in solutions via policies (such as scaling and routing), and high availability functions built into the Web Application pattern type. For Virtual Applications these features are fully integrated and optimized functions as are all elements of Virtual Applications. However, similar features have also been available for WebSphere Application Deployments in Virtual System patterns with a special extension.
WebSphere Virtual Enterprise provides application virtualization for traditional WebSphere ND solutions and this same feature is delivered for Virtual System pattern deployments of WebSphere Application Server by use of the Intelligent Management Pack. Leveraging the capabilities of Workload Deployer with Virtual Systems lets you gain the benefits of server virtualization and to reduce hardware, provide rapid and consistent deployment of entire systems, dynamically adjust resource consumption, and much more. Leveraging the capabilities of the Intelligent Management Pack provides the ability to manage service level agreements with elastic scaling and health management, lower operational costs, and provide for improved application management. These two solutions together provide a powerful combination to improve the management and resiliency of your enterprise applications.
If you would like to learn more about application virtualization using the Intelligent Management Pack in conjunction with Virtual System Patterns in IWD then please join Keith Smith and myself tomorrow for a webcast on this very topic. Keith is the lead architect on our WebSphere Virtual Enterprise and Intelligent Management Pack products and brings a wealth of experience in this space. So don't miss this opportunity - register here.
One of the things that often comes up at some point in IBM Workload Deployer conversations is the notion of self-service access. Specifically, users want to know what the appliance provides that enables them to allow various teams in their organization to directly deploy the middleware environments they need. In other words, they want to use IBM Workload Deployer to tear down the traditional barriers that exist between those that request the environment and those that fulfill said request. Now, as we begin to elaborate on this notion, it becomes quickly apparent that in order to effectively enable self-service, IBM Workload Deployer must deliver a few things.
First, IBM Workload Deployer must provide the means to define users with various levels of access. Second, IBM Workload Deployer must provide the means to define resource access at a fine-grained level to different users and groups of users. Check and check. The appliance has been doing this since the beginning of WebSphere CloudBurst. Without those two things, the conversation of self-service access would end pretty quickly. However, there is a final capability that is equally important: IBM Workload Deployer must deliver a means to limit resource consumption at a fine-grained level.
In IBM Workload Deployer there are a couple of ways to achieve this. First, you could define multiple cloud groups and allow access to those groups in a way that maps directly to resource entitlements. While that may work in some situations, others call for even more granularity. You may want to allow multiple different users or groups to access a cloud group, but you may want to allow different consumption limits for each of these groups. In this situation, you can take advantage of environment profiles and a new option when defining users of IBM Workload Deployer.
Consider the case that you have a group of developers and you want to limit their consumption of memory in the cloud. First, you start by defining your development users and for each you select Environment Profile Only as the value for the Deployment Options field.
By selecting the above value for the deployment options of a user, you restrict that user to only deploying via an environment profile as opposed to general cloud group deployments. After defining all of your development users, you may choose to organize them into a user group for easier management. At that point, you can define environment profiles and determine which ones your developers should have access to using the Access granted to field of the profile.
Within the environment profile, you can define resource consumption limits for compute resource and software licenses. For instance, you can define a limit on the amount of virtual memory consumed by all deployments using the profile. It is important to note that the limit is cumulative for ALL deployments that use the profile.
Now that all of the controls are in place, consider the deployment process for one of your development users. They pick a virtual system pattern, click the deploy icon and begin to configure the pattern for deployment. In the Choose Environment section of the deployment dialog, your development user will only be able to select the Choose profile option for deployment. Further, they will only be able to deploy using the environment profiles to which they have access.
After the deployment completes, a look at the Environment limits section in the profile shows the current usage totals.
Now suppose another development user, or even the same one, comes along and attempts to deploy another virtual system pattern even though the profile limits have already been reached. The user can initiate the deployment, but they will get a near immediate failure owing to the fact that they would exceed consumption limits if the deployment were allowed to proceed.
The same kind of enforcement occurs regardless of the resource limit type. You can use this approach to limit the consumption of CPU, virtual memory, storage, or software licenses among the various different users or groups of users you define in IBM Workload Deployer. If you combine fine-grained resource consumption limits with varying permissions and fine-grained access, I think you are on the road to truly enabling self-service in the enterprise.
Customers are always impressed when they learn about the simplicity, resiliency, and rapid time to value they can received from virtual applications. However, they are usually a little mystified at how virtual applications really work. After all - they have become quite accustomed to doing things the "traditional way" where they control every aspect of their applications manually. Virtual Applications represent an entirely new way of thinking. Sure, the benefits are enormous but can you really trust them? How is it doing all of this anyway?
What seems like "magic" is really a sophisticated and coordinated set of activities driven and coordinated by IBM Workload Deployer while leveraging the expertise built into the pattern type. Yes, you can trust it because experts have worked to build the system and created to it react and respond much faster than you can. When moving away from manual processes to automated processes it is always nice to get a sense of what is really happening. I think it is just human nature. We can't really place our trust in something until we have first hand experience or understand what it is really doing ... I guess it is the critic inside each one of us. Even after you've experienced the value it is still reassuring to see and understand the "how".
It is the "how does it do that?" type of question that I attempted to answer for virtual applications in a blog post I wrote on the Expert Integrated Systems blog recently. It attempts to pull the curtain aside and describe what is actually happening to support a virtual application pattern. As with my previous post - this was written for IBM PureApplication Systems but the concepts are 100% applicable to IBM Workload Deployer. I think you will find it interesting ... Continue reading ...
Script packages are an integral part of virtual system patterns in IBM Workload Deployer. By attaching script packages to your patterns, you provide customizations particular to your unique cloud-based middleware environments. Customizations provided by script packages might include installing applications, creating application resources, integrating with external enterprise systems, and much more. The bottom line is, if you are creating virtual system patterns, you will almost certainly be creating script packages.
Largely, the act of creating a script package is independent of IBM Workload Deployer. The appliance does not dictate a particular scripting language, so all you need to do is make sure you can invoke your logic in the operating system environment. Your script package may be a wsadmin script, shell script, Java program, Perl script, and on and on. After you create the actual contents of your script package, you will then load that asset into the IBM Workload Deployer catalog.
Once loaded into the catalog, you define several attributes of your script package, including the executable command, command arguments, variables, execution time, and more. The process for defining these attributes is trivial using the intuitive UI in IBM Workload Deployer, but I wanted to take a little time to remind you of a technique I recommend to all users defining script packages. You can actually package a JSON file within the script package that defines all of the script's attributes. The format of the file is simple, and I am including an example below:
The example above is one taken from a script package in our samples gallery, and it shows the basics of which you need to be aware. Notice that in the JSON file, you can provide a name, description, unzip location, executable command, command arguments, variables, and more. You only need to ensure that the name of this JSON file is cbscript.json and that you include it at the root of the script package archive. Once you have done that, you load the script package archive into the catalog, refresh the script package details, and voila -- all the attribute definitions appear!
You may ask why I recommend this since it could seem like an unnecessary step. My answer to that is that you have to define these attributes anyway, so you might as well capture it once in the file. Once you capture it once in the file, you can ensure that if the same script needs to be reloaded, or if you need to move it to another appliance, its definition will be exactly the same (and presumably correct). I use this approach for all of my work, and for all of the samples I contribute to our gallery, and it really saves me a lot of misplaced effort that can result from typos. If you are out there creating script packages, try adopting this approach. I'm pretty sure you will be happy you did!
Lately, I have run into multiple situations where an IBM Workload Deployer user has been trying to decide exactly how they want to create their customized images for the cloud. Essentially, they have been trying to decide whether to use the native extend and capture capabilities of IBM Workload Deployer, or to pursue the use of the Image Construction and Composition Tool (also included with the appliance). The conversations have been interesting and challenging, but more importantly, they have been a reminder that constructing enterprise-ready environments for the cloud does not happen by magic. It takes thought, deliberate planning, sustainable design, and the tools to carry everything out.
The tools part we have covered. I have every confidence, bolstered by user experience after user experience, that IBM Workload Deployer and associated tools (like the Image Construction and Composition Tool) equip you to build highly customized, cloud-based application environments. In this post, I want to focus in on the thought process that goes into how you decide to build your customized environment. Specifically, I would like to talk about important points to consider as you try to understand whether to use the native extend and capture capabilities of IBM Workload Deployer or the Image Construction and Composition Tool.
To be clear from the outset, I am not trying to provide a decision flowchart in this post. For all intents and purposes, that would be next to impossible. Instead, I want to pose to you some important questions that you should ask of yourself, along with the reasons why I believe those queries to be important. Keeping in mind that this is not an all-inclusive list, here it goes:
Question: Are the customizations that you want to make congruent with an IBM-supplied image?
Reason: One of the first decisions you should make is whether or not you can start with an IBM-supplied image as the base for your customization. You need to know what middleware elements (type and version) make up your environment and what operating system should host that environment (version and distribution). You can match that information against the list of content that IBM supplies. If there is a match, you should start by looking at extend and capture to customize that image to meet your needs. If there is no direct match, you may be looking at the Image Construction and Composition Tool.
Question: Does your custom content supplement middleware content supplied in an IBM image?
Reason: If you simply need to add additional components that supplement software already in an IBM image, I believe it is best to first examine the use of extend and capture. Whether these components are IBM software or not is irrelevant as the extend and capture functionality does not care.
Question: How configurable do you want to make the custom content in your image?
Reason: If you are adding content into the image, you need to think about just how configurable you need it to be. When you use extend and capture, you add the content to an existing image in a manner that pretty well ends up being opaque to IBM Workload Deployer. To configure that content, you need to have script packages and make sure they are part of every pattern you create based on the image. Alternatively, if you use the Image Construction and Composition Tool, you can embed configuration behavior in the image's activation engine, and you can expose deploy-time parameters without needing to include script packages in every single pattern. As an example, if you need to add a monitoring agent into your environment, you would likely do this via extend and capture and end up with a pretty simple script package to configure that agent during deployment. If however, you need to create an image with a custom database, you would likely favor the Image Construction and Composition Tool as you could embed common deploy-time configuration parameters directly in the image. For a database, there are likely to be many more deploy-time configuration parameters that you want to expose as compared to a more simple monitoring agent.
Question: Is your main focus on making operating system changes?
Reason:If your primary focus is on making operating system changes AND the answer to the first question is that your target content aligns well with IBM-supplied images, then extend and capture is where you want to start. Of course, you need to make sure that you can make all necessary changes to the OS with extend and capture, but I will say that this capability is not very restrictive at all.
Admittedly, this is a short list, but I believe it is a good starting point for how you decide upon one approach versus the other. Also, I would be remiss not to point out that these tools are absolutely not mutually exclusive. Many users I work with use a combination of the two approaches. In fact, there are some use cases that call for both tools. Start by creating a completely custom image in the Image Construction and Composition Tool, and then subject that image to the extend and capture process in IBM Workload Deployer to customize it for a particular purpose, team, project, etc. I hope you find this helpful, and I welcome your feedback or thoughts!
When IBM Workload Deployer v3.0 rolled around, the appliance introduced the concept of shared services. These were services that a cloud administrator could launch into the cloud infrastructure defined to IBM Workload Deployer, and use to serve a number of different application deployments. There were, and continue to be, two main shared services: a proxy service and a cache service. The shared proxy service does pretty much what you may guess. It provides request routing capabilities across multiple different instances of multiple different applications, thereby providing a centralized resource that encapsulates this basic need in an application environment. You can probably also guess what the caching service does. It caches things! Specifically, in IBM Workload Deployer v3.0 it provided in-memory caching of HTTP sessions, thus ensuring high availability of data stored in those sessions.
Undoubtedly, the ability to make HTTP session data fault tolerant is extremely critical in any application environment, cloud-based environments included. However, the applicability of a shared cache service is much further reaching, and in IBM Workload Deployer v3.1, we are starting to open this service up to your applications. What does this mean to you? Quite simply it now means that you can access this cache directly from your application code. If you are familiar with WebSphere eXtreme Scale or the DataPower XC10 Caching Appliance, then you know exactly what I mean. You can use the WebSphere eXtreme Scale ObjectGrid API to insert, read, update, and delete entries that exist in the in-memory cache. The underlying cache technology is based on the same code that powers WebSphere eXtreme Scale and DataPower XC10, so you can be sure that your cache is scalable, fault tolerant, responsive, and otherwise able to meet the needs of your application.
As I hope you find to be the case with many IBM Workload Deployer capabilities, this is a superbly simple capability to leverage. When you deploy virtual application patterns based on the IBM Workload Deployer Pattern for Web Applications, the capability is simply there. The underlying runtime that is serving your application is automatically augmented with the capabilities necessary so that your applications can connect to and utilize the deployed caching service. It is also worth pointing out that you can utilize the caching capabilities provided by this shared service for applications and application infrastructure that you deploy via virtual system patterns as well. You can either choose to augment the WebSphere Application Server environment with the XC10 Feature Pack (a deploy-time option for virtual system patterns built on WebSphere Application Server Hypervisor Edition v8), or you can configure WebSphere Application Server as you always would when integrating with a WebSphere eXtreme Scale environment or a DataPower XC10 Appliance.
What's the real benefit to all of this you ask? Well, when you use the shared caching service, you get the benefits of a distributed, in-memory, extremely scalable cache without having to deal with too much setup or administration. You simply tell IBM Workload Deployer how many resources you want to dedicate to your cache, and deploy the shared service. IBM Workload Deployer takes care of the details, including scaling in and out the cache to meet the needs of the system. On top of all of this, there is also an option to configure 'Next to the Cloud' caching. If you currently own DataPower XC10 appliances, you can make those available to virtual application pattern deployments (this was already possible with virtual system patterns) by simply providing details of the location of the appliance collective in question.
Put simply, setting up, administering, and utilizing an object caching service for your applications has never been easier. Check it out and let us know what you think!
We've been talking a lot recently about Virtual Application Patterns and enhancements to this deployment model in IBM Workload Deployer v3.1. This is appropriate because virtual applications are a substantial evolution for application deployment in a private cloud. Virtual Application Patterns deliver on the promise of Platform-as-a-Service - letting you focus on the application while Workload Deployer builds the necessary platform to deploy and manage your application.
However, Virtual System Patterns are still alive and well ... and quite frankly, this is where many people begin to explore the functionality provided in Workload Deployer. For many, it is a logical first step to start recreating familiar physical environments in the private cloud and then leverage these environments to develop and test their applications. It is also a great way to build out new applications using familiar concepts, leveraging existing scripts, and take full advantage of the agility, consistency, and increased resource utilization available in a Workload Deployer managed private cloud.
You may recall that virtual system patterns are sometimes called topology patterns because they are used to define a topology middleware configuration to meet application requirements. With a virtual system pattern you define exactly the type of middleware configuration that you need for your application environment and Workload Deployer provisions exactly that configuration when the pattern is deployed to your private cloud.
To use an automotive analogy, you might compare virtual systems to building your own hot-rod from a molded frame while virtual applications are more like purchasing a complete vehicle from a dealer. When you purchase a vehicle from a dealer you receive a fully functional automobile. Sure, you can choose the color and some options – but you don't necessarily know the details of all of the components that make your vehicle functional. Just add a driver (you) and off you go! This saves you substantial time and money while freeing you from the need to be an automotive engineer. As with the production vehicle, virtual applications are optimized for a specific purpose and are extremely effective when used for that purpose. All you need to do is add your application (the driver) and run-time requirements. Virtual system patterns are like the hot-rod approach. You start with a modeled frame of sorts (hypervisor edition images) – thereby saving time and effort so you don't have a start from scratch. However you still have the responsibility and flexibility to create a very unique custom vehicle. Doing so requires more expertise and a greater time investment when compared to a production vehicle (virtual application), but you get to decide all of the details. With virtual systems you specify the exact vehicle you need for your application. This provides substantial flexibility but requires a deep knowledge of the middleware and an investment of time building necessary scripts and other elements to support your application environment.
So as I mentioned, virtual system patterns are very popular. And if you have been following recent posts about the enhancements delivered in IBM Workload Deployer v3.1 you noticed that several of the features primarily focused on virtual applications have at the same time been extended to virtual system patterns - such as the shared caching service and the new base AIX image. So we certainly consider virtual systems deployment model to be important. IBM Workload Deployer v3.1 delivered new hypervisor edition images and the IBM Image Construction and Composition Tool was bundled with Workload Deployer - primarily used for creating custom images to leverage in virtual system patterns. The IBM Image Construction tool is a substantial advancement in the ability to create your own custom base images.
To help communicate that we haven't been neglecting virtual system deployment patterns, I created a new demo to highlight this deployment model. The demo begins by providing a quick overview of the components that go into a virtual system pattern. It then shows how to clone a pattern to customize it for your own purpose, deploy it, monitor licenses, and monitor resource usage in your private cloud. Finally, it shows a quick demonstration of installing an emergency fix to a deployed virtual system instance.
I'll be showing this and other demos at IBM Pulse 2012 next week. I hope to see you there!
Virtual Application Patterns are one of the major new features in IBM Workload Deployer v3. You've heard this concept discussed on this blog before and it is really a revolutionary way to manage your applications in a private cloud environment. With Virtual Application Patterns you provide declarative information about your application including functional and non-functional requirements of that application. You get to focus on the application rather than the middleware configuration and IBM Workload Deployer takes care of all the details necessary to launch your application with the criteria you specify. This application-centric approach radically simplifies the deployment of applications in a private cloud. And it is not just the deployment that is simplified; it is also the monitoring, metering, logging, security, caching, etc ... that is consolidated and simplified as well. Everything is custom tailored for the particular application type to provide a significant level of integration and optimization for elastic, efficient, multi-tenant, automated management and execution of that application workload.
In IBM Workload Deployer v3 there are two different types of virtual application patterns provided out of the box; a pattern for web applications and a pattern for database applications. It's no accident that these are also the two most heavily utilized types of applications in most enterprises. Of course more patterns will be appearing in the future and you have the opportunity to create your own custom patterns ... but these first two patterns can cover a substantial number of current application workloads.
So why am I introducing all of this again? Well, I want to make you aware of a new article that was just published which covers virtual application patterns in a very consumable way with enough detail and screen shots to get you started down this path. It is appropriately named: Easy virtual app automation using Workload Deployer . It really does a great job of covering not only the web application pattern - but it also introduces the database pattern (DBaaS) and shared services. If you are about to embark on virtual applications this is a great place to start.
Applications - just like humans, animals, plants, and many other things - have a life cycle. They are conceived, given birth, grow, do foolish things in youth, hopefully improve over time, have problems that need to be fixed, don't always age well .... and eventually they will die and release their assets to the next generation. Sounds kind of familiar, doesn't it?
One of the many benefits of virtual application patterns in IBM Workload Deployer and related IBM offerings is support for the complete life cycle of the application. You can manage the complete life cycle of virtual applications from a single interface that is fully integrated and well thought out - not just a series of links from one product UI to a different product UI. This eliminates the complexity of having to work with different interfaces, paradigms, metaphors, controls, labels, names, authorization, and so on - that is often the norm in many customer environments today. I think the benefits of this integration are obvious - eliminating confusion, configuration, miscommunication, interpretation, and mapping errors. Providing a truly complete solution also facilitates a common knowledge base and encourages cooperation and collaboration among teams. You can share patterns, providing consistent governance for a solution, guarantee consistency in deployments, and build upon the expertise provided by others. Having an integrated solution for design, development, deployment, configuration changes, monitoring, and problem determination ensures that time is not wasted and valuable information is not lost.