Lately, I have run into multiple situations where an IBM Workload Deployer user has been trying to decide exactly how they want to create their customized images for the cloud. Essentially, they have been trying to decide whether to use the native extend and capture capabilities of IBM Workload Deployer, or to pursue the use of the Image Construction and Composition Tool (also included with the appliance). The conversations have been interesting and challenging, but more importantly, they have been a reminder that constructing enterprise-ready environments for the cloud does not happen by magic. It takes thought, deliberate planning, sustainable design, and the tools to carry everything out.
The tools part we have covered. I have every confidence, bolstered by user experience after user experience, that IBM Workload Deployer and associated tools (like the Image Construction and Composition Tool) equip you to build highly customized, cloud-based application environments. In this post, I want to focus in on the thought process that goes into how you decide to build your customized environment. Specifically, I would like to talk about important points to consider as you try to understand whether to use the native extend and capture capabilities of IBM Workload Deployer or the Image Construction and Composition Tool.
To be clear from the outset, I am not trying to provide a decision flowchart in this post. For all intents and purposes, that would be next to impossible. Instead, I want to pose to you some important questions that you should ask of yourself, along with the reasons why I believe those queries to be important. Keeping in mind that this is not an all-inclusive list, here it goes:
Question: Are the customizations that you want to make congruent with an IBM-supplied image?
Reason: One of the first decisions you should make is whether or not you can start with an IBM-supplied image as the base for your customization. You need to know what middleware elements (type and version) make up your environment and what operating system should host that environment (version and distribution). You can match that information against the list of content that IBM supplies. If there is a match, you should start by looking at extend and capture to customize that image to meet your needs. If there is no direct match, you may be looking at the Image Construction and Composition Tool.
Question: Does your custom content supplement middleware content supplied in an IBM image?
Reason: If you simply need to add additional components that supplement software already in an IBM image, I believe it is best to first examine the use of extend and capture. Whether these components are IBM software or not is irrelevant as the extend and capture functionality does not care.
Question: How configurable do you want to make the custom content in your image?
Reason: If you are adding content into the image, you need to think about just how configurable you need it to be. When you use extend and capture, you add the content to an existing image in a manner that pretty well ends up being opaque to IBM Workload Deployer. To configure that content, you need to have script packages and make sure they are part of every pattern you create based on the image. Alternatively, if you use the Image Construction and Composition Tool, you can embed configuration behavior in the image's activation engine, and you can expose deploy-time parameters without needing to include script packages in every single pattern. As an example, if you need to add a monitoring agent into your environment, you would likely do this via extend and capture and end up with a pretty simple script package to configure that agent during deployment. If however, you need to create an image with a custom database, you would likely favor the Image Construction and Composition Tool as you could embed common deploy-time configuration parameters directly in the image. For a database, there are likely to be many more deploy-time configuration parameters that you want to expose as compared to a more simple monitoring agent.
Question: Is your main focus on making operating system changes?
Reason:If your primary focus is on making operating system changes AND the answer to the first question is that your target content aligns well with IBM-supplied images, then extend and capture is where you want to start. Of course, you need to make sure that you can make all necessary changes to the OS with extend and capture, but I will say that this capability is not very restrictive at all.
Admittedly, this is a short list, but I believe it is a good starting point for how you decide upon one approach versus the other. Also, I would be remiss not to point out that these tools are absolutely not mutually exclusive. Many users I work with use a combination of the two approaches. In fact, there are some use cases that call for both tools. Start by creating a completely custom image in the Image Construction and Composition Tool, and then subject that image to the extend and capture process in IBM Workload Deployer to customize it for a particular purpose, team, project, etc. I hope you find this helpful, and I welcome your feedback or thoughts!
As I have mentioned before, IBM Workload Deployer v3.0 introduces choices in pattern-based deployment models. One of those models, virtual system patterns, is a carry over from the WebSphere CloudBurst Appliance. When you use virtual system patterns in IBM Workload Deployer, you can take advantage of all of the techniques you put to use in WebSphere CloudBurst. This is certainly good news for current WebSphere CloudBurst users, but it goes a bit further. Instead of simply maintaining the status quo with virtual system patterns, which would have been reasonable considering the introduction of virtual application patterns, we chose to continue to expand on your customization options for this pattern deployment model. In particular, I want to discuss three new features in IBM Workload Deployer that may help you to better construct and manage virtual system patterns.
The first new feature is one that I have been eagerly awaiting. In the new version of the appliance, we provide you with the ability to specify part and script package ordering in your pattern. This means that, within the virtual system pattern editor, you can tell IBM Workload Deployer in which order to start the virtual machines in your pattern, and you can specify in which order to invoke the script packages within the pattern during deployment. This eliminates the need for special script invocation orchestration logic in your pattern (I had customers resorting to a semaphore like approach using a shared file system), and it allows you to be more declarative about the virtual machine bring-up process. There are constraints, specifically with the part ordering. Some images will impose an implied part start-up order that you cannot change. For instance, deployment manager parts in the WebSphere Application Server Hypervisor Edition image must start before custom node parts. The good news is the pattern editor will not allow you to specify a part start-up order that violates these constraints. The image below shows an example of the ordering view in the virtual system pattern editor.
Another new feature that may influence the way you build virtual system patterns is the introduction of Add-Ons. You can think of Add-Ons as special script packages that you can include in your virtual system pattern that perform system-level configuration actions. Specifically, you can include add-ons in your virtual system pattern to add an operating system user, add a virtual disk, or add a NIC during the deployment process. You include Add-Ons in your pattern by simply dragging and dropping them onto a part in your pattern, just as you do with script packages today. The difference between script packages and Add-Ons is that IBM Workload Deployer will ensure the invocation of all Add-Ons before any other scripts run during deployment. We include default Add-On implementations for adding a user, disk, and NIC.
The last new feature I want to talk about today has more to do with how you manage or govern the deployment of virtual system patterns. In WebSphere CloudBurst 2.0, we introduced the idea of Environment Profiles as a way to extend your customization reach into the deployment process. Initially, these profiles gave you the ability to directly assign IP addresses to virtual machines in your deployment, declaratively specify virtual machine naming formats, and easily split a single pattern deployment across multiple cloud groups. In IBM Workload Deployer, you will be able to use these same profiles to set resource consumption limits for pattern deployments. In particular, you will be able to set cumulative limits for virtual CPU, memory, storage, and software licenses used by deployments tied to a specific profile, thereby giving you finer-grained control over cloud resource consumption. The picture below shows the new resource limit aspects of environment profiles.
Virtual system patterns are key in the deployment model choices for IBM Workload Deployer. Not only did we carry the concept over from WebSphere CloudBurst to IBM Workload Deployer, but we made it even better. Expect this trend to continue!
Among the major features of the new virtual application pattern in IBM Workload Deployer is the notion of elasticity. That is, as your application needs more resources, it gets them. When your application can meet its SLAs with fewer resources, the environment shrinks. With this kind of pattern, you enable elasticity by specifying a policy and defining the scaling trigger (i.e. CPU usage, application response times, database response times, etc.). What may have been a bit lost in some of these new announcements regarding IBM Workload Deployer is the fact that you can now leverage this core feature of cloud, elasticity, in your virtual system patterns.
If you have read this blog in the past, you probably already know that the Intelligent Management Pack is an option for virtual system patterns built using WebSphere Application Server Hypervisor Edition. When you enable the Intelligent Management Pack option, you are essentially building and deploying WebSphere Virtual Enterprise (WVE) environments. For those of you not familiar with WVE, the best way to describe it is that it provides you with application and application infrastructure virtualization capabilities. Of its many capabilities, one most germane to our discussion today is the ability for users to attach SLAs to applications and then have WVE automatically prioritize requests and manage resources in order to meet those SLAs. Inherent in this capability is the ability to dynamically start and stop application server processes (JVMs) as required. In other words, WVE provides JVM elasticity.
The fact that WVE provides JVM elasticity is nothing new. Further, IBM Workload Deployer started providing virtual machine (VM) elasticity in previous versions (when it was WebSphere CloudBurst). With this feature, you could add or remove VMs to an already deployed virtual system using dynamic virtual machine operations provided by the appliance. The catch was that the VM elasticity was a manual action and you could not link this elasticity to the same SLAs tied to your applications. Well, thanks to a new feature in WebSphere Virtual Enterprise and easy integration provided by the Intelligent Management Pack, this is no longer the case.
Starting in IBM Workload Deployer 3.0, you can take advantage of a new WVE feature called Elasticity Mode when using the Intelligent Management Pack. Elasticity mode is not unique to IBM Workload Deployer, but a concept new to the base WVE product. It allows one to define actions for how WVE should grow and shrink the set of nodes used by application server resources. Like the basic JVM elasticity capability in WVE, these node elasticity actions trigger based on SLAs tied to your applications. Consider the case that you are using elasticity mode and your application is not currently meetings its SLA. If WVE does not think it can start any more application server instances on the current set of nodes, it will grow the set of nodes per your elasticity configuration. Conversely, if WVE detects that it can meet SLAs with fewer nodes, it will shrink the resources per your elasticity configuration.
In IBM Workload Deployer, using elasticity mode becomes even easier. All you need to do is use the Intelligent Management Pack and enable the elasticity mode option in your virtual system patterns. When you do this, you get automatic integration between IBM Workload Deployer and the deployed WVE environment. What does that mean? It means that if WVE detects it needs more nodes, it will automatically call back into IBM Workload Deployer and request that the appliance provision a new VM that will serve as a node for application server processes. It also means that if WVE detects it could meet SLAs with fewer resources, it will call into IBM Workload Deployer and ask it to remove a node. All of this happens without any user scripting. All you have to do is enable this option in your patterns and configure SLAs appropriate for your applications.
To me, this exciting new feature brings out the best of elasticity capabilities in both IBM Workload Deployer and WebSphere Virtual Enterprise. The result is a single management plane that gives you both VM and JVM elasticity for your cloud-based application environments. Best of all, elasticity actions map directly to SLAs for your applications. After all, when it comes to cloud, it's the application that really matters!
Sorry for the late notice - but I just realized that I hadn't blogged about a webcast that I am participating in tomorrow (Tuesday, 9/13)!
Chris Brealey (a Senior Technical Staff Member and Rational Enterprise Architect) and I are hosting an InformationWeek WebCast tomorrow (Tuesday, 9/13) entitled "Quickly and Efficiently Design, Develop, Deploy, and Test Workload Application Patterns to Save Months and Millions". I encourage you to register now for this free event (or if you can't make it tomorrow listen to it at your convenience as it will be recorded ... but you still need to register).
I'm really looking forward to this webcast. IBM Workload Deployer's predecessor, WebSphere Cloudburst Appliance, delivered unmatched capabilities for middleware deployments and management using Virtual System patterns (topology) - delivering complete middleware topologies in a rapid, consistent, and repeatable fashion. This has greatly improved the ability of development and test organizations to meet the ever increasing demands of today's agile development processes in addition to the assurance it provides for production environments. All of that value is still present (and improved) in IBM Workload Deployer but there is even more value in the new Virtual Application Patterns, as we've mentioned in previous posts.
Virtual Applications build upon this same notion of consistency and speed found in Virtual Systems while at the same time introducing a radical simplification to hosting your applications. Using an application-centric, declarative approach with Virtual Applications (workloads) it is even easier to deliver your applications rapidly leaving Workload Deployer to ensure the middleware environment is constructed and optimized to meet your application criteria. Virtual Applications usher IBM Workload Deployer into the realm of Platform-as-a-Service ... with even greater simplicity and agility to host your application in the most efficient fashion. As with Virtual System patterns earlier, we expect the introduction of Virtual Applications to continue to improve the dev/test lifecycle as well as production. The robust capabilities of Rational Application Developer and the simplicity of Virtual Application patterns in Workload Deployer make for a great combination.
I will start off the webcast with a discussion of PaaS and IBM Workload Deployer Virtual Application patterns. Chris will then discuss the application development process and how that is influenced with the introduction of the cloud environment. Chris will then explore the integration that is available in Rational Application Developer for IBM Workload Deployer. Finally, we will walk through a scenario that demonstrates how to leverage Virtual Application patterns in IBM Workload Deployer to design a solution that is then shared with the developer. Using Rational Application Developer the developer delivers the application into the pattern and moves it to test and finally pre-production. We will end with a question and answer time. I hope you can join us as we explore how we can use these technologies to increase agility and efficiency.
We've been talking a lot recently about Virtual Application Patterns and enhancements to this deployment model in IBM Workload Deployer v3.1. This is appropriate because virtual applications are a substantial evolution for application deployment in a private cloud. Virtual Application Patterns deliver on the promise of Platform-as-a-Service - letting you focus on the application while Workload Deployer builds the necessary platform to deploy and manage your application.
However, Virtual System Patterns are still alive and well ... and quite frankly, this is where many people begin to explore the functionality provided in Workload Deployer. For many, it is a logical first step to start recreating familiar physical environments in the private cloud and then leverage these environments to develop and test their applications. It is also a great way to build out new applications using familiar concepts, leveraging existing scripts, and take full advantage of the agility, consistency, and increased resource utilization available in a Workload Deployer managed private cloud.
You may recall that virtual system patterns are sometimes called topology patterns because they are used to define a topology middleware configuration to meet application requirements. With a virtual system pattern you define exactly the type of middleware configuration that you need for your application environment and Workload Deployer provisions exactly that configuration when the pattern is deployed to your private cloud.
To use an automotive analogy, you might compare virtual systems to building your own hot-rod from a molded frame while virtual applications are more like purchasing a complete vehicle from a dealer. When you purchase a vehicle from a dealer you receive a fully functional automobile. Sure, you can choose the color and some options – but you don't necessarily know the details of all of the components that make your vehicle functional. Just add a driver (you) and off you go! This saves you substantial time and money while freeing you from the need to be an automotive engineer. As with the production vehicle, virtual applications are optimized for a specific purpose and are extremely effective when used for that purpose. All you need to do is add your application (the driver) and run-time requirements. Virtual system patterns are like the hot-rod approach. You start with a modeled frame of sorts (hypervisor edition images) – thereby saving time and effort so you don't have a start from scratch. However you still have the responsibility and flexibility to create a very unique custom vehicle. Doing so requires more expertise and a greater time investment when compared to a production vehicle (virtual application), but you get to decide all of the details. With virtual systems you specify the exact vehicle you need for your application. This provides substantial flexibility but requires a deep knowledge of the middleware and an investment of time building necessary scripts and other elements to support your application environment.
So as I mentioned, virtual system patterns are very popular. And if you have been following recent posts about the enhancements delivered in IBM Workload Deployer v3.1 you noticed that several of the features primarily focused on virtual applications have at the same time been extended to virtual system patterns - such as the shared caching service and the new base AIX image. So we certainly consider virtual systems deployment model to be important. IBM Workload Deployer v3.1 delivered new hypervisor edition images and the IBM Image Construction and Composition Tool was bundled with Workload Deployer - primarily used for creating custom images to leverage in virtual system patterns. The IBM Image Construction tool is a substantial advancement in the ability to create your own custom base images.
To help communicate that we haven't been neglecting virtual system deployment patterns, I created a new demo to highlight this deployment model. The demo begins by providing a quick overview of the components that go into a virtual system pattern. It then shows how to clone a pattern to customize it for your own purpose, deploy it, monitor licenses, and monitor resource usage in your private cloud. Finally, it shows a quick demonstration of installing an emergency fix to a deployed virtual system instance.
I'll be showing this and other demos at IBM Pulse 2012 next week. I hope to see you there!
If you are reading this blog then I am pretty sure that you are interested in the agility that can be achieved by rapidly provisioning middleware systems and standing up virtual applications in a private cloud environment. However there are other aspects of agility that you should also consider. One such aspect is the ability to build applications that can be easily maintained, updated, and extended. This is where OSGi technology comes into the picture.
If you have been working with the IBM Workload Deployer (or watching some IBM Workload Deployer demos) you may have noticed a category of components in the virtual application builder called OSGi Components.
Maybe you already know all about OSGi applications and the value they bring to an enterprise. Or, perhaps you noticed this and decided that you would search for some more information on this odd acronym and just what an OSGi application is all about.
In a nutshell OSGi technology is a way to define dynamic modules for Java. It provides a standard way to encapsulate components (called bundles) with metadata that define versioned package dependencies, service dependencies, packages exported, services exported, etc... basically everything you need to know about this bundle so that it can be connected up with other bundles to support a particular solution. These bundles can then be grouped together into applications and dynamically wired to fulfill necessary dependencies at runtime. The OSGi framework provides all of the necessary capability to manage the dependencies and resolve any problems.
Those who leverage OSGi technology benefit from improved time-to-market and reduced development costs. The loose coupling provided by the OSGi framework reduces maintenance costs and facilitates the dynamic delivery of components in a running system. Of course there's a lot more to it than just that ... involving portability across different environments, achieving the appropriate level of isolation or sharing within an environment, and integrating with the many different technologies and patterns already available today. I don't think I know enough about OSGi to do it justice here. But fortunately for me (and you) there are several experts who can make it all clear.
One such expert is Graham Charters and there is a great opportunity to hear him introduce this topic and also participate in a dialogue about the concepts and what they mean for your business. Graham will be leading a Global WebSphere Community Lab Chat on Wednesday of this week (July 20th) entitled: How can OSGi make your enterprise more agile. Graham is the IBM technical lead in the OSGi Alliance Enterprise Expert Group and an active participant in the open source community implementing many of these standards. So register now for this free session and learn how OSGi can make your enterprise even more agile.
Applications - just like humans, animals, plants, and many other things - have a life cycle. They are conceived, given birth, grow, do foolish things in youth, hopefully improve over time, have problems that need to be fixed, don't always age well .... and eventually they will die and release their assets to the next generation. Sounds kind of familiar, doesn't it?
One of the many benefits of virtual application patterns in IBM Workload Deployer and related IBM offerings is support for the complete life cycle of the application. You can manage the complete life cycle of virtual applications from a single interface that is fully integrated and well thought out - not just a series of links from one product UI to a different product UI. This eliminates the complexity of having to work with different interfaces, paradigms, metaphors, controls, labels, names, authorization, and so on - that is often the norm in many customer environments today. I think the benefits of this integration are obvious - eliminating confusion, configuration, miscommunication, interpretation, and mapping errors. Providing a truly complete solution also facilitates a common knowledge base and encourages cooperation and collaboration among teams. You can share patterns, providing consistent governance for a solution, guarantee consistency in deployments, and build upon the expertise provided by others. Having an integrated solution for design, development, deployment, configuration changes, monitoring, and problem determination ensures that time is not wasted and valuable information is not lost.
In my opinion, declarative deployment models are key to the entire notion of Platform as a Service (PaaS). That is, users should concern themselves with what they want, but not necessarily how to get it. The PaaS system should be able to interpret imperatives from the user and automatically convert that to a running system. In this respect, I think the new virtual application pattern, and more specifically policies, in IBM Workload Deployer takes a giant leap toward a more declarative deployment model.
In IBM Workload Deployer, policies allow you to 'decorate' your virtual application pattern with functional and non-functional requirements. In other words, they provide a vehicle for you to tell the system what qualities of service you expect for your application environment. To put a little context around this discussion, let's examine the policies available in the virtual application pattern for web applications. Specifically, let's look at the four policy types you can attach to Enterprise Application, Web Application, and OSGI Application components in this pattern:
Scaling policy: When it comes to cloud, the first thing many folks think about is autonomic elasticity. Applications should scale up and down based on criteria defined by the user. Well, that is exactly what the scaling policy lets you do. You simply attach this policy to your application component, and then specify properties that define when to scale. First, you choose a scaling trigger from a list that includes application response time, CPU usage, JDBC connection wait time, and JDBC connection pool usage. After choosing your trigger, you decide the minimum and maximum number of application instances for your deployment, and then you choose the minimum number of seconds to wait for an add or remove action. At this point, you can deploy your application and IBM Workload Deployer will monitor the environment, automatically triggering scaling actions as needed.
JVM policy: I would be willing to bet that nearly all of you tune the JVM environment into which you deploy your applications. The JVM policy allows you to take two common tuning actions, setting the JVM heap sizes and passing in JVM arguments, as well as attach a debugger to the Java process (especially useful in development and test phases). You can also use the policy to enable verbose garbage collection (invaluable to understanding heap usage patterns for your application) and select the bit level (from 32 or 64) for your application. Again, all you have to do is attach the policy and specify the properties. IBM Workload Deployer will take care of the required configuration updates.
Routing policy: The routing policy provides a simple way to specify virtual hostnames and allowable protocols (HTTP or HTTPS) for your application. Attach the policy, provide the virtual hostname you want to use, select the desired protocols, and that's it! Remember, once you set the virtual hostname you will need to update your name server to map the hostname to the appropriate IP address.
Log policy: During the development and test phase, it is likely that you will want to enable certain trace strings in the application runtime. The log policy allows you to provide trace strings for your application, and it makes sure that the appropriate configuration updates occur in the deployed environment.
While this is not an exhaustive explanation of each of the policies above, I hope it gives you a basic idea of what they are and how to use them. To me, declarative deployment models are going to be a crucial part of making PaaS successful, so I am really excited about the notion of policies in IBM Workload Deployer. What do you think?
Customization capabilities have been very important to the design of IBM Workload Deployer going back to the beginning with WebSphere CloudBurst. Having the ability to quickly spin up environments in a cloud really does little good if those environments are not customized according to your needs. If you look at the virtual system pattern capability, it is why we always had the notion of custom images, custom patterns, and custom scripts. We give you a strong foundation, and you tweak it here and there to create what you want.
Customization is not a concept unique to virtual system patterns. The virtual application model in IBM Workload Deployer supports many different mechanisms for you to tailor your cloud-based environments. You can start with the virtual application pattern types that we ship and use any components in those patterns to build a custom environment. The patterns you build can include your own configuration (within the set of configurable parameters) and include policies that you need for your environment. In looking at just the IBM Workload Deployer Pattern for Web Applications and the IBM Workload Deployer Pattern for Databases, there are quite a number of scenarios you can support with your cloud. However, what happens when you want to go a little further and color outside the lines of what we provide?
At some point you may have heard or read that the entire virtual application pattern model resides on a pluggable architecture. In effect, this means that everything about a virtual application pattern type, from the elements that show up when building a pattern to the management interface you interact with after deployment, is customizable. The fundamental unit of customization for a virtual application pattern type is a plugin. Plugins provide the know-how in terms of installing, configuring, integrating, and managing the application types supported by a given pattern. Plugins also provide metadata that control what users see as they build and manage these patterns. In short, plugins are the source of truth for virtual application patterns!
If you looked in IBM Workload Deployer, you would find the collection of plugins that support the virtual application pattern types shipped with the offering. While that is interesting, you should also know that you can supply your own plugins. That's right. You can develop a plugin, and load it directly into the appliance. This allows you to do two very important things. First, you can extend the virtual application pattern types that come with IBM Workload Deployer with any kind of functionality you deem important. This may be additional monitoring, integration with external systems, or any number of other extensions. Second, you can create new virtual application pattern types that support your desired workloads. You can support the workloads with the software of your choosing so long as you can supply the necessary know-how in your plugins. In either case, you contribute the plugin, and your customized components become first class members of the IBM Workload Deployer landscape.
Okay, so I admit that this is not necessarily news. We have supported user-contributed plugins since the release of IBM Workload Deployer. However, there is something new that significantly lowers the barrier to entry in the custom plugin game. Early last week, IBM announced the IBM Workload Plugin Development Kit. This kit provides a set of tools and samples designed to make the construction and packaging of custom plugins a simple process. In my opinion, this reiterates our commitment to an extensible, application-centric cloud approach, and it represents a huge step forward in the industry as a whole. Be sure to check this out, and don't be shy with the comments and feedback!