When many people think of cloud computing they immediately think of virtualization and virtual machines in particular. This is completely natural and not at all surprising. After all, one of the core underlying technologies necessary for cloud computing is virtualization. However, it is important not to confuse one element of cloud computing with the entire thing - and this can sometimes happen. Many people have begun to leverage virtual machines in their on premise environment and sometimes begin to call this their private cloud. While virtualization is a substantial step forward and help gets you started down the necessary path of standardization and automation that is essential in a cloud - it is not in and of itself "a cloud".
The National Institute of Standards and Technology has published its definition of cloud computing. This is a very complete and yet concise definition that includes not only the essential characteristics of a cloud solution but also the service models (IaaS, PaaS, SaaS) and deployment models (public, private, hybrid, community). It is a great way to get a perspective on cloud and can be useful when considering the solutions of various vendors.
Let me summarize the essential elements of cloud from this definition here:
broad network access
So, this is interesting. Not only is this much more than just virtualization - but virtualization isn't even mentioned in the list explicitly. Not to worry - virtualization is of course important and is included under the resource pooling topic. I would assert that virtualization is also necessary to facilitate the type of on-demand, self-service, elastically scaling resources that are leveraged in a cloud. What is crystal clear from this definition is that there is a lot more to a cloud solution than just virtual images and some hypervisor infrastructure upon which to run them. Somebody must provide the necessary on-demand/self-service capabilities, the network access to these services, the management of the resource pools, enabling true elasticity for running systems, measuring services and so forth. IBM Workload Deployer provides just such capabilities for the on-premise cloud allowing you to efficiently deploy patterns built for virtual systems and virtual applications with deep knowledge of the middleware that is being provisioned to optimize these solutions. Furthermore, Workload Deployer provides the complete lifecycle management from pattern creation, to deployment and provisioning, applying maintenance, resource and license management in the on-premise cloud, elastic scalability, and eventually returning resources to the on-premise cloud to be reused. Workload Deployer is a complete solution for not only server virtualization but of course for cloud computing.
However, virtualization doesn't have to stop with just virtual machines. It is a general principle that can be applied to more than just servers. At its core, virtualization is really about providing a level of abstraction between some real resources and the consumers of those resources. This is a natural fit when we think of server virtualization and virtual machines. However, there are also substantial benefits to be gained by adopting a similar abstraction between the middleware and the applications themselves - sometimes referred to as application virtualization.
By application virtualization I mean providing the capabilities to abstract the application from the underlying infrastructure such that it can be elastic, participate in health management policies, and provide agility across the pool of application infrastructure resources. This type of application virtualization is built into our Virtual Application pattern (hence the name) in Workload Deployer and surfaced in solutions via policies (such as scaling and routing), and high availability functions built into the Web Application pattern type. For Virtual Applications these features are fully integrated and optimized functions as are all elements of Virtual Applications. However, similar features have also been available for WebSphere Application Deployments in Virtual System patterns with a special extension.
WebSphere Virtual Enterprise provides application virtualization for traditional WebSphere ND solutions and this same feature is delivered for Virtual System pattern deployments of WebSphere Application Server by use of the Intelligent Management Pack. Leveraging the capabilities of Workload Deployer with Virtual Systems lets you gain the benefits of server virtualization and to reduce hardware, provide rapid and consistent deployment of entire systems, dynamically adjust resource consumption, and much more. Leveraging the capabilities of the Intelligent Management Pack provides the ability to manage service level agreements with elastic scaling and health management, lower operational costs, and provide for improved application management. These two solutions together provide a powerful combination to improve the management and resiliency of your enterprise applications.
If you would like to learn more about application virtualization using the Intelligent Management Pack in conjunction with Virtual System Patterns in IWD then please join Keith Smith and myself tomorrow for a webcast on this very topic. Keith is the lead architect on our WebSphere Virtual Enterprise and Intelligent Management Pack products and brings a wealth of experience in this space. So don't miss this opportunity - register here.
To continue with the series of blog posts regarding WebSphere CloudBurst FAQs, I want to take a look at one aspect of the deployment process.
When you leverage WebSphere CloudBurst to push patterns (complete WebSphere Application Server configurations) into a private cloud, the appliance provides an advanced placement algorithm to determine exactly where the resulting WebSphere virtual systems will reside. It attempts to match the needs of the pattern to the correct set of hypervisors that have been defined. WebSphere CloudBurst considers things like storage, CPU, memory, and high availability characteristics when placing the pattern, and this is all done by the appliance without you having to intervene at all.
This is certainly nice in that it absolves you from having to make such placement decisions. Having said this though, you may be thinking of a question that comes up quite often:
If WebSphere CloudBurst controls the placement of the pattern, how can I make sure that certain deployments end up on certain servers (hypervisors)?
Considering what I just told you above, it may not seem that it's possible to control what machines end up hosting your virtual system since the appliance takes care of that placement for you. However, the organized use of WebSphere CloudBurst cloud groups allows you to take advantage of the intelligent placement provided by the appliance while retaining a level of control over which machines end up hosting particular deployments.
In WebSphere CloudBurst all patterns are deployed to cloud groups. Cloud groups are a collection of hypervisors that have been defined within the appliance. The basic deployment mapping is depicted in the image below:
As seen above, you can create a cloud group for any purpose (dev, test, QA, production, etc.), including any hypervisors that you desire as long as a given hypervisor only belongs to a single cloud group. When you are ready to deploy a pattern, you simply select the cloud group you want to deploy to:
By selecting a cloud group for deployment, you are implicitly selecting the physical machines that will host your deployment. The cloud group could consist of anywhere from one to N hypervisors, so you are afforded the ability to restrict the location of your virtual systems as necessary.
I hope this helped explain a little bit about cloud groups in WebSphere CloudBurst. If you're looking for more information about WebSphere CloudBurst cloud groups, I'd also suggest you watch this video on our YouTube channel.
Over time, many of our users learn to effectively leverage WebSphere CloudBurst user roles and fine-grained access controls to map activities and responsibilities in the appliance to the appropriate people and teams within their organization. Using these controls, they are able to define actions that a user or group can take, and they can define the set of resources on which they can take those actions. It is efficient, flexible, and an absolute necessity in many enterprise scenarios.
In some cases though, I talk with users that want a little more control, or probably better put, governance over the actions a user can take within a given role. Most often, this need arises when the discussion of pattern authoring comes up. If you want a user in WebSphere CloudBurst to be able to create patterns, you simply give them the Create new patterns permission. Once you give them that permission, the user can create patterns using both virtual image parts and script packages in the catalog. For many of the users I talk with, this approach suits their needs.
However, in some scenarios administrators want a little more insight and control over how the pattern authors build their patterns. Specifically, they want to ensure that patterns contain only approved virtual image parts and script packages. While you can certainly use the fine-grained access controls of the appliance to expose only the 'approved' virtual image parts and script packages, that alone may not be enough. After all, the definition of what is 'approved' may be different when building a pattern for testing purposes versus one built for production purposes. If the same pattern author builds both of those patterns, fine-grained access controls do not help as much. So, what can you do?
Have I ever told you how much I love the WebSphere CloudBurst CLI? It's powerful, easy to use, and a great automation enabler. It is also the perfect tool for our problem above. If you are looking to enforce certain constraints on WebSphere CloudBurst patterns, I strongly recommend using the CLI as a governance tool.
To provide a concrete example of what I mean, let's take a look at a generic pattern checking script I am working on now (I hope to have this in the samples gallery soon). Consider the case that I want to check that all of my test patterns for a specific application environment contain 1 deployment manager and between 1 and 3 custom nodes. In addition, I want to make sure that the parts for these nodes come from an approved virtual image, and I want to verify that the deployment manager contains the correct application installation script package. With the script I am currently writing, you would start by encapsulating this information in a properties file.
PatternAssertion_1=Customer Processing Test Environments
PatternAssertion_1_Requirements=Deployment manager:1:415:Install customer process app;Custom nodes:1-3:415
In the above, the PatternAssertion_1 key provides a name for the pattern verification assertion. The PatternAssertion_1_Requirements key provides the requirements for the pattern. The above requirements indicate that for a pattern to meet the assertion, it must contain 1 deployment manager part from the virtual image with reference number 415. In addition, the deployment manager must contain a script named Install customer process app. A valid pattern must also contain 1 to 3 custom node parts, also based on the virtual image with reference number 415. When done defining my requirements in the properties file, I simply invoke a script and pass in the file. As a result, I get information about which patterns satisfy or do not satisfy the assertions. For example:
The Customer Process Application pattern satisfied the requirements of the Customer Processing Test Environments assertion.
OR The Customer Process Application pattern did not satisfy the requirements of the Customer Processing Test Environments assertion
due to the following reason: The pattern is required to have a minimum of 1 and a maximum of 3 Custom node part(s), but it had 4.
As I said, I hope to have this sample script posted to the samples gallery soon. I am going through some final revisions and enhancements that I hope make it better and more generally applicable. In the meantime, I wanted to point out that pattern governance is indeed doable, and in fact not very hard to achieve with the CLI. I will be sure to post an update when the sample script is ready. In the meantime, let me know if you have any questions or comments.
When it comes to IBM Workload Deployer, I have no illusions concerning our competitors. They are out there, and they are constantly on the attack. Their dubious claims aside, I know this because I still get asked quite frequently to explain the benefits of IBM Workload Deployer versus some other general purpose cloud provisioning and management solution. So, while I have done that many times in various forums, I figured it was time to address the subject here on the blog.
When comparing IBM Workload Deployer to the other available solutions, I honestly feel comfortable saying we have no direct competition. I know you believe me to be biased, and rightly so, but let me explain why I think the competition is much more perception than reality. To do this, I want to focus on the patterns-based approach that IBM Workload Deployer takes to cloud provisioning and management.
Let's start with virtual system patterns in IBM Workload Deployer. Virtual system patterns allow you to build and deploy completely configured and integrated middleware environments as a single unit. These patterns build on top of our special IBM Hypervisor Edition images that bottle up the installation and quite a bit of the configuration of the underlying middleware products. Further, when using virtual system patterns, IBM Workload Deployer manages and automates the orchestration of the integration tasks that need to happen to setup a meaningful middleware environment. For instance, when deploying WebSphere Application Server you do not need to do anything on your end to deploy a clustered, highly available environment. When deploying WebSphere Process Server in this manner, you do not need to take any administrative actions to produce a golden topology. You just deploy patterns and the images, patterns, and appliance take care of the rest. Of course, you can add your own customizations and tweaks in the pattern, but we take care of the common administrative actions that would otherwise require your care.
I am not sure of a better way to say it, so I will be blunt: When deploying products delivered in IBM Hypervisor Edition form, no other solution compares to the virtual system pattern capability offered by IBM Workload Deployer. It is not even close. Can you provision products like WebSphere Application Server or WebSphere Portal using other cloud provisioning tools? Sure, but you should be aware that you will be writing and maintaining your own installation, configuration, and integration scripts. It is also likely that you will end up developing a custom interface through which deployers request your services (something not necessary when using the slick IBM Workload Deployer UI). All of this takes time, resource, and money. More importantly, this is not differentiating work and distracts from the real end goal: serving up applications. IBM Workload Deployer can deliver this operational capability right out of the box, and it can do so in a way that costs less than custom developed and maintained solutions.
When considering IBM Workload Deployer versus the competition, it is also important to consider the new virtual application pattern capability delivered in version 3.0. The virtual application pattern capability is a testament to IBM's thought leadership in and commitment to cloud computing for middleware application environments. Virtual application patterns take a bold step forward in raising the level of abstraction beyond the middleware environment and up to the most important resource in enterprise environments: the application. With a virtual application pattern, you simply provide your application and specify both functional and non-functional requirements for that application. When ready, you deploy that pattern, and IBM Workload Deployer sets up the necessary middleware infrastructure and deploys the provided application. Moreover, the appliance will monitor and autonomically manage the environment (i.e. scale it up and down) based on the policies you specify. Quite simply, this is a deployment and management capability our competition cannot match.
There is more to consider than just patterns though. The appliance makes it really simple to apply maintenance and upgrades to environments running in your cloud. It can autonomically manage your deployed environments (through policies in virtual application patterns and the Intelligent Management Pack for virtual system patterns), and it effectively abstracts the underlying infrastructure of your cloud environment. This abstraction is the reason IBM Workload Deployer can deploy your environments to PowerVM, zVM, and VMware environments. It also makes it easy to deploy the same environment to multiple different underlying platforms, thus accommodating typical platform changes that happen as an application moves from development to production. The best part of all is that the deployer’s experience is the same regardless of the underlying infrastructure since the appliance hides any platform idiosyncrasies.
The bottom line is that the appliance is purpose built to deploy and manage middleware and middleware application environments in a cloud, and as such, delivers immense out-of-the-box and ongoing value in this context. I should also point out that the design of the appliance acknowledges its purposeful nature. The CLI and REST API interfaces allow you to integrate the appliance into the operations of those general purpose provisioning solutions. In this way, IBM Workload Deployer acts as a middleware accelerator for your cloud computing efforts. This means that if you do have a general purpose solution, IBM Workload Deployer can still provide considerable value and let you avoid developing a considerable subsystem dedicated to deployment and management of middleware in the cloud. We believe in this type of integration, and have in fact built it into our own IBM solutions.
I could go on and on differentiating IBM Workload Deployer from the competition, but I hope my comments above give you a good context on why I think the appliance is in a league of its own. Of course, I always appreciate comments and feedback, so don't be shy!
I recently read a post by David Linthicum in which he proposes that a key benefit of cloud computing is the ability to transfer risk from the enterprise to the cloud provider.
At first glance, this seems an obvious benefit of using a public cloud for computing resources. Cloud providers take care of the onerous task of providing computing resources across an organization. If the resources need to be updated, require critical maintenance, or need emergency action, the cloud provider will provide those services. Enterprise IT departments are left to devote effort toward delivering technological capabilities to the business. However, does any of this imply a transfer of risk?
I'd answer that question with "It depends." Whether or not an enterprise has transferred risk by contracting with a public cloud provider depends on the provisions in the Service Level Agreement (SLA) that exists between the enterprise and provider. In some cases (maybe most) the SLA simply provides a refund for a portion of the service fee based on the impacted services. This is clearly not a case of transference of risk. The loss of current and new business sustained by the enterprise during the service outage is not indemnified by the cloud provider. In this sense, the enterprise has done nothing more than transfer the management of their risks to a third party.
True risk transference can be achieved, but it means that SLAs provide both service fee refunds and business loss indemnification. During a service outage, an enterprise's risk is not the fee they are paying for the service but instead the impact on current and future profits. There must be stipulations in the SLAs to address these losses for risk transfer to have taken place.
The differences between transferring risk and risk management may seem obvious, but it does serve to underscore the importance of SLAs in the cloud computing world. Enterprises need to fully understand these SLAs in order to accurately assess the benefits of using a cloud proider. SLAs are poised to be critical in the cloud computing world, and I'm interested to see how they will help shape the competitive landscape of the industry.
The reason I suggest the application proxy approach is twofold. First, it affords you the ability of having custom interactions with the REST API. For instance, you may insert logic into the server-side proxy code that returns only a subset of the JSON data contained in the response from the appliance. Alternatively, in an effort to reduce the chattiness on your client-side, you may join JSON data from multiple different REST requests to the appliance to fulfill a single client request. You may even decide to represent the data in an all together different format than JSON. All of these options and many more are available to you if you implement an application-based proxy to the REST API.
The second reason I suggest the application approach is that it is easier, and seemingly safer, to not deal with user passwords on the client-side. If you setup your application proxy, you can configure it to retrieve the appropriate password from a secure location (like an encoded file) based on information passed along in the request. This means the password information is only present in the request (in encoded form of course) from the application proxy to the WebSphere CloudBurst Appliance.
The good news about the application-based proxy approach is that it is simple to put in place. I composed one using the open source Apache Wink project. The Apache Wink project is an open source implementation of the JAX-RS specification (and then some), and it enables you to develop POJOs that are in turn exposed in a RESTful manner. In my case, I had a single resource POJO:
The Apache Wink runtime routes any HTTP GET request whose path is like /resources/* to the getResources method in the WCAResource class. This method passes along information taken from the query string (the host name of the target WebSphere CloudBurst Appliance and the requesting WebSphere CloudBurst username), as well as the HTTP path information and sends it on to the getResource method declared as follows:
The getResource method above uses the WebSphere CloudBurst host name and the request path to construct the URL for the corresponding WebSphere CloudBurst REST API call. Next, it constructs an Apache Wink Resource object and sends the REST request along to the WebSphere CloudBurst Appliance. How do we authenticate this request? We use the WebSphere CloudBurst username (sent as a query string parameter) to retrieve the appropriate encoded password information. Once we have that, we construct the necessary header for basic authorization over SSL.
The application-based proxy shown here is simply a pass-through. It does not manipulate the data returned from the WebSphere CloudBurst REST API, nor does it map a single client-side call to multiple REST requests. However, it would be simple enough to extend it to do any of those things. If you have any questions about the code here, please let me know. I'd be happy to share more of the code, or talk about how and where to extend it.
I write a lot about WebSphere CloudBurst script packages. Typically, I write about what they are, how to create them, and even provide some samples from time to time. I find that most of the time I'm either writing or talking about script packages from the standpoint that they allow you to automate the delivery of customizations to environments you deploy with WebSphere CloudBurst. More specifically, I usually explain how you can include these script packages in your patterns to ensure that your custom scripts execute as part of every pattern deployment. The truth is, that is not the whole story. In fact, it's only 1/3 of the story.
In WebSphere CloudBurst, when you define a script package you also define its execution mode. The execution mode can be one of three values, and it indicates the invocation time for the script. The default value is at virtual system creation, and that tells WebSphere CloudBurst to automatically invoke the script as part of the deployment process. This seems to be the most commonly used execution mode, and in the original version of WebSphere CloudBurst it was the only available mode (which probably attributes to why I only usually tell 1/3 of the story here). As you may expect, there is a wide range of usage scenarios for this class of script packages including installing applications, activating monitoring agents, registering cells with an externally managed DNS server, and much more.
If you are like me (and many humans), you enjoy and actually expect symmetry. In that regard, it probably comes as no surprise that there is a script package execution mode called at virtual system deletion. As the name indicates, this class of script packages executes as an automatic part of the virtual system deletion process. When a user tells WebSphere CloudBurst to remove a virtual system, before it shuts down the machines in the system, it will run each script package marked to execute at virtual system deletion. Typical use cases for these scripts include removing information about the cell from externally managed DNS servers, freeing up connections with external systems, and other external 'clean up' activities.
So this leaves the final execution mode for script packages, the when I initiate it mode to be precise. This class of script packages executes when explicitly triggered by a user. In the virtual machine detail section for a deployed virtual system, you can see a list of user-initiated script packages for a given machine. There is a start button by each of the user-initiated script packages that allows you to invoke the script when, and as many times as you need to. While these script packages have many different use cases, the most common use case is to deploy application updates. Users build these application update scripts, attach them to a pattern, and invoke them whenever they want to deliver an updated application into their already deployed environment.
WebSphere CloudBurst script packages are one of the main vehicles for delivering your customizations to your cloud environments. The three execution modes mentioned above allow you to determine when the right time to deliver those customizations is.
Customers are always impressed when they learn about the simplicity, resiliency, and rapid time to value they can received from virtual applications. However, they are usually a little mystified at how virtual applications really work. After all - they have become quite accustomed to doing things the "traditional way" where they control every aspect of their applications manually. Virtual Applications represent an entirely new way of thinking. Sure, the benefits are enormous but can you really trust them? How is it doing all of this anyway?
What seems like "magic" is really a sophisticated and coordinated set of activities driven and coordinated by IBM Workload Deployer while leveraging the expertise built into the pattern type. Yes, you can trust it because experts have worked to build the system and created to it react and respond much faster than you can. When moving away from manual processes to automated processes it is always nice to get a sense of what is really happening. I think it is just human nature. We can't really place our trust in something until we have first hand experience or understand what it is really doing ... I guess it is the critic inside each one of us. Even after you've experienced the value it is still reassuring to see and understand the "how".
It is the "how does it do that?" type of question that I attempted to answer for virtual applications in a blog post I wrote on the Expert Integrated Systems blog recently. It attempts to pull the curtain aside and describe what is actually happening to support a virtual application pattern. As with my previous post - this was written for IBM PureApplication Systems but the concepts are 100% applicable to IBM Workload Deployer. I think you will find it interesting ... Continue reading ...
I was very encouraged by the consistently positive response we got at IMPACT for our WebSphere CloudBurst and Rational Automation Framework for WebSphere (RAFW) integration. I believe there were many reasons for this response: accelerated time to value, decreased investment needs for activities that are not core to your business, lowered barrier of entry for provisioning and configuring WebSphere cells, and much more. While those are certainly all very real and valuable benefits, I also believe that quite a bit of interest in this integrated solution comes from the fact that it is applicable to a number of needs common to you, our WebSphere users.
With that in mind, let's look at some (not all) of the scenarios where WebSphere CloudBurst and RAFW integration can help you:
Create WebSphere CloudBurst patterns that include configuration without scripting: Users love our WebSphere CloudBurst patterns. They really see the value in codifying both the topology and configuration of their application infrastructure. However, some users do not have existing WebSphere configuration scripts and do not have the time and/or resource to invest in creating these scripts. They are looking for a solution that provides not only the provisioning of WebSphere environments but also the configuration of said environments (configuration beyond what our IBM Hypervisor Edition images encapsulate, specifically application-oriented configuration). RAFW provides this capability in the form of 500+ out-of-the-box configuration actions for WebSphere environments. This includes actions to create JDBC resources, create JMS queues, deploy applications, configure application containers, and much, much more. You can create WebSphere CloudBurst patterns that contain a special script package, which points back to a RAFW project containing a set of configuration actions. During deployment, WebSphere CloudBurst will provision your WebSphere environment and then cause the invocation of the specified RAFW project, which in turn runs a set of configuration actions against the provisioned environment. This means you can set up full-blown, ready-to-go application environments with absolutely no user-supplied scripting. In fact, I took this approach to setup a J2EE performance benchmark application, DayTrader 2.0, running on WebSphere Application Server. Those of you familiar with the application know this is not a trivial environment to stand up. Yet, I did it without having to personally write a single line of configuration scripting, and it was all ready to go in around thirty minutes.
Creating WebSphere CloudBurst patterns from existing environments: This comes up all the time. I go through a standard introduction to WebSphere CloudBurst, users see the value, love the patterns-based approach, and immediately want to know how they get their existing environments into the form of a pattern. RAFW, along with the special WebSphere CloudBurst script package, can make this a straightforward and hardened process. You use a capability in RAFW to import the configuration of an existing cell, thereby creating a RAFW environment for that configuration. You then create a WebSphere CloudBurst pattern with a topology congruent to your existing environment, attach the special script package I mentioned earlier, and you are done with the import! When you deploy this pattern, you simply specify the RAFW environment that you created earlier (the one that holds the configuration data for your existing environment) and a RAFW project that will apply the configuration data in that RAFW environment to the WebSphere environment provisioned by WebSphere CloudBurst. The creation of the WebSphere environment, as well as its configuration, happens in a completely automated fashion.
Configure, capture, reuse: There are many situations that may require you to make manual changes to a WebSphere cell after it has been deployed. For example, during performance testing for your application, you might discover that you need to tweak the number of available threads in the web container. As another example, for the first setup of a given application environment, you may want to quickly deploy the cell using WebSphere CloudBurst and then manually install and configure your applications to make sure everything is just right. In either case, it is likely that you want to capture the updated configuration and make sure that any future deployments use those updates. Again, WebSphere CloudBurst and RAFW makes this simple. First, you build a pattern that encapsulates your WebSphere topology (the types and quantity of nodes you want) and attach the special script package mentioned above. For the first deployment, you simply specify the name of the new RAFW environment you want to create. Once the system is up, you log into the WebSphere administration console, make your necessary customizations, and then you use RAFW to import that updated configuration thus updating the initially created RAFW environment. For subsequent deployments, you simply deploy the same pattern, specifying the same RAFW environment as well as a RAFW project, which RAFW automatically created for you during the first deployment. This project applies the configuration (the one you manually established and imported into RAFW) to the WebSphere environment setup by WebSphere CloudBurst.
Configure WebSphere environments across virtual and physical settings: It seems that in many cases our users manage the same WebSphere environment across both virtual and physical settings. For example, they may provision the application environment using WebSphere CloudBurst for everything from development to pre-production, and then for production provision that same environment to a set of physical servers. At least, they try to provision the same environment. In reality, it is tough to reproduce the exact same configuration once you break from the WebSphere CloudBurst patterns-based approach. However, if you stored the configuration of your WebSphere cell as a RAFW environment, you could apply that configuration data to a WebSphere cell regardless of whether it existed in the physical or virtual world. Once you move to physical, you do lose out on the fast provisioning, WebSphere intelligence, cloud management capabilities, and automated integration with RAFW that you get when using WebSphere CloudBurst, but if it is in your process to move to physical hardware at some point, reusing the same RAFW environment certainly eases the migration task.
I hope this sheds some light on some of the common issues WebSphere CloudBurst and the Rational Automation Framework for WebSphere can combine to solve really well. This is by no means an exhaustive list, but really meant to point out the broad application of the solution. If you want to see how it works, check out this video.
I was at a customer meeting the other day, and someone asked me if they could query WebSphere CloudBurst for an inventory of all of their virtual system deployments. This person was of course aware that he could go to the web console and very quickly view all of the virtual systems. What he wanted though was something that he could run to generate a report that contained all of this information. For a purpose like this, harnessing the WebSphere CloudBurst CLI is exactly the way to go.
I thought I'd write a simple CLI script that provides an example of how you could do this.
from datetime import datetime
outFile.write("WebSphere CloudBurst Virtual System Inventory\n")
outFile.write("Total virtual systems: " + str(len(cloudburst.virtualsystems)))
def writeVSDetails(outFile, virtualSystem):
outFile.write("\tVirtual system name: " + virtualSystem.name)
outFile.write("\tCreated from pattern: " + virtualSystem.pattern.name)
outFile.write("\tVirtual system status: " + virtualSystem.currentstatus_text)
created = datetime.fromtimestamp(virtualSystem.created)
outFile.write("\tVirtual system creation date: " + created.strftime("%B %d, %Y %H:%M:%S"))
outFile.write("\tTotal virtual machines: " + str(len(virtualSystem.virtualmachines)))
def writeVMDetails(outFile, virtualMachine):
outFile.write("\t\tVirtual machine name: " + virtualMachine.name)
outFile.write("\t\tVirtual machine display name: " + virtualMachine.displayname)
outFile.write("\t\tCreated from image: " + virtualMachine.virtualimage.name)
outFile.write("\t\tVirtual machine hypervisor: " + virtualMachine.hypervisor.name + " | " + virtualMachine.hypervisor.address)
outFile.write("\t\tVirtual machine IP address: " + virtualMachine.ip.ipaddress)
outFileLoc = sys.argv
outFile = open(outFileLoc, 'w')
for virtualSystem in cloudburst.virtualsystems:
for virtualMachine in virtualSystem.virtualmachines:
As a result of invoking this script using the CLI's batch mode, content is written to the file location supplied by the caller.
WebSphere CloudBurst Virtual System Inventory
Total virtual systems: 3
Virtual system name: Single server
Created from pattern: WebSphere single server
Virtual system status: Started
Virtual system creation date: January 15, 2010 16:37:20
Total virtual machines: 1
Virtual machine name: Standalone 0
Virtual machine display name: Single server cbvm-110 default
Created from image: WebSphere Application Server 22.214.171.124
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
Virtual system name: Development WAS Cluster
Created from pattern: Custom WAS Cluster - Development
Virtual system status: Started
Virtual system creation date: January 18, 2010 14:08:46
Total virtual machines: 2
Virtual machine name: DMGR 0
Virtual machine display name: Development WAS Cluster cbvm-112 dmgr
Created from image: WebSphere Application Server 126.96.36.199
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
Virtual machine name: Custom Node 1
Virtual machine display name: Development WAS Cluster cbvm-111 custom
Created from image: WebSphere Application Server 188.8.131.52
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
Virtual system name: DB2 for development use
Created from pattern: DB2
Virtual system status: Started
Virtual system creation date: January 18, 2010 14:09:58
Total virtual machines: 1
Virtual machine name: DB2 Enterprise Server 32bit Trial 0
Virtual machine display name: DB2 for development use cbvm-113
Created from image: DB2 Enterprise 184.108.40.206 32-bit Trial
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
I withheld IP addresses and host names above for obvious reasons, but if you ran the script against your environment you would see actual host name and IP address values. The script above is written once, and it can be subsequently run anytime you want an inventory of virtual systems running in your WebSphere CloudBurst cloud. There's other information available for virtual systems and virtual machines that I didn't show here, and you can retrieve it if necessary for your inventory report. In addition, I chose to print this information as regular text in a file supplied by the caller, but you might choose to generate the report in another format including XML, JSON, or anything else for that matter.
-- Dustin Amrhein
p.s. As with any sample code or script I provide here, the above is only a sample and offered as-is.
Among the major features of the new virtual application pattern in IBM Workload Deployer is the notion of elasticity. That is, as your application needs more resources, it gets them. When your application can meet its SLAs with fewer resources, the environment shrinks. With this kind of pattern, you enable elasticity by specifying a policy and defining the scaling trigger (i.e. CPU usage, application response times, database response times, etc.). What may have been a bit lost in some of these new announcements regarding IBM Workload Deployer is the fact that you can now leverage this core feature of cloud, elasticity, in your virtual system patterns.
If you have read this blog in the past, you probably already know that the Intelligent Management Pack is an option for virtual system patterns built using WebSphere Application Server Hypervisor Edition. When you enable the Intelligent Management Pack option, you are essentially building and deploying WebSphere Virtual Enterprise (WVE) environments. For those of you not familiar with WVE, the best way to describe it is that it provides you with application and application infrastructure virtualization capabilities. Of its many capabilities, one most germane to our discussion today is the ability for users to attach SLAs to applications and then have WVE automatically prioritize requests and manage resources in order to meet those SLAs. Inherent in this capability is the ability to dynamically start and stop application server processes (JVMs) as required. In other words, WVE provides JVM elasticity.
The fact that WVE provides JVM elasticity is nothing new. Further, IBM Workload Deployer started providing virtual machine (VM) elasticity in previous versions (when it was WebSphere CloudBurst). With this feature, you could add or remove VMs to an already deployed virtual system using dynamic virtual machine operations provided by the appliance. The catch was that the VM elasticity was a manual action and you could not link this elasticity to the same SLAs tied to your applications. Well, thanks to a new feature in WebSphere Virtual Enterprise and easy integration provided by the Intelligent Management Pack, this is no longer the case.
Starting in IBM Workload Deployer 3.0, you can take advantage of a new WVE feature called Elasticity Mode when using the Intelligent Management Pack. Elasticity mode is not unique to IBM Workload Deployer, but a concept new to the base WVE product. It allows one to define actions for how WVE should grow and shrink the set of nodes used by application server resources. Like the basic JVM elasticity capability in WVE, these node elasticity actions trigger based on SLAs tied to your applications. Consider the case that you are using elasticity mode and your application is not currently meetings its SLA. If WVE does not think it can start any more application server instances on the current set of nodes, it will grow the set of nodes per your elasticity configuration. Conversely, if WVE detects that it can meet SLAs with fewer nodes, it will shrink the resources per your elasticity configuration.
In IBM Workload Deployer, using elasticity mode becomes even easier. All you need to do is use the Intelligent Management Pack and enable the elasticity mode option in your virtual system patterns. When you do this, you get automatic integration between IBM Workload Deployer and the deployed WVE environment. What does that mean? It means that if WVE detects it needs more nodes, it will automatically call back into IBM Workload Deployer and request that the appliance provision a new VM that will serve as a node for application server processes. It also means that if WVE detects it could meet SLAs with fewer resources, it will call into IBM Workload Deployer and ask it to remove a node. All of this happens without any user scripting. All you have to do is enable this option in your patterns and configure SLAs appropriate for your applications.
To me, this exciting new feature brings out the best of elasticity capabilities in both IBM Workload Deployer and WebSphere Virtual Enterprise. The result is a single management plane that gives you both VM and JVM elasticity for your cloud-based application environments. Best of all, elasticity actions map directly to SLAs for your applications. After all, when it comes to cloud, it's the application that really matters!
If you've read anything I've written about WebSphere CloudBurst up to this point you know all about patterns. Using the appliance you can easily and quickly build, deploy, and manage these representations of your middleware application environments. Today, I want to focus in on the deployment piece in particular and take a look at how you can easily automate this process.
You can use the WebSphere CloudBurst web console to deploy patterns, and when doing so you can even schedule the deployment to happen at a later date. This scheduling capability certainly gets you on the road to an automated deployment process, but what if you want to take it one step further and eliminate the need for someone to login and manually move around the web console to schedule automated deployments? In this case, you can use either the CLI or the REST interface that WebSphere CloudBurst offers.
In this post I thought I'd take a look at using the CLI interface in order to set the stage for some nice automation around pattern deployment. It starts out with a properties file that provides details about my deployment. This includes the cloud to deploy to, the pattern to deploy, password information, and the time at which the virtual system should start.
SYSTEM_NAME_PREFIX=New App Development
TARGET_CLOUD=Default ESX group
TARGET_PATTERN=WebSphere single server
Imagine that the properties file above gets written as the result of some other action, such as the completion of your application's build process. With the properties file in place, and I'll point out that your properties file can and probably will be more robust than above, let's move on to the code that handles the deployment process based on the information in said file. First, we have a small amount of CLI code to retrieve and parse the input data (I omitted the straight-forward properties retrieval for space):
from datetime import datetime, timedelta
from java.util import Properties
from java.io import FileInputStream
// read in and retrieve properties using java.util.Properties API (i.e. props.getProperty('DEPLOYMENT_DATE'))
parsedParts = deploymentDate.split(" ")
systemName = systemName + "_" + deploymentDate
dateParts = parsedParts.split("/")
timeParts = parsedParts.split(":")
monthPart = int(dateParts)
dayPart = int(dateParts)
yearPart = int(dateParts)
hourPart = int(timeParts)
minutePart = int(timeParts)
Next is the code that actually schedules the pattern deployment:
First we get the desired deployment time and current time as datetime objects. After that, assuming the desired deployment time has not already elapsed, we calculate the difference between the desired deployment time and current time. This difference, in seconds, is then added to the result of the time.time() value to come up with a start time. After that is done, we simply retrieve the cloud that was indicated in the properties file, and then we call the runInCloud method for the pattern indicated. When calling the runInCloud method we supply the name of the virtual system that will be created, password information, and the start time we calculated earlier. As a result of this method call, a task will be generated in the target WebSphere CloudBurst Appliance and the virtual system will be started at the specified time. This will happen in an automated fashion with no human intervention required.
That's really all there is to automating the pattern deployment process using the CLI. In a more complete, end-to-end scenario you may envision the completion of one process, such as an application build process mentioned above, result in the writing of the properties file and in turn the call into the CLI to deploy a pattern. As always, feel free to send me any comments or questions.
A while back I co-authored an article along with Chris Ahl from Tivoli and Ken Klingensmith from WebSphere Technical Sales about the customization of virtual images in WebSphere CloudBurst. In the article we approached image customization as a means to enable IBM Tivoli Monitoring for the operating system within virtual machines dispensed by WebSphere CloudBurst. Today I posted a short demonstration that discusses and shows this particular integration scenario. If you are interested, but haven't had time to read the article, you may want to watch the video first as it should give you a good overview of the process and results.
Talk of Tivoli reminds me that IBM Pulse 2010 is just around the corner. I'll be going to discuss WebSphere CloudBurst and how it can be paired with software from IBM Tivoli for high-value integration scenarios. In the session I'll be talking about the Tivoli Monitoring integration as well as other key points such as our integration with Tivoli Service Automation Manager, IBM CloudBurst, and more. The best part about the session is that I will be co-presenting alongside a WebSphere CloudBurst customer that will dole out practical advice for using WebSphere CloudBurst within the enterprise. Join us on Tuesday February 23rd from 3:30 - 4:30 in Conference Center 306.
Remember, any time you have questions about WebSphere CloudBurst please pass them along. You can leave comments on this blog, or you can reach me at my new Twitter location @damrhein.
Script packages are an integral part of virtual system patterns in IBM Workload Deployer. By attaching script packages to your patterns, you provide customizations particular to your unique cloud-based middleware environments. Customizations provided by script packages might include installing applications, creating application resources, integrating with external enterprise systems, and much more. The bottom line is, if you are creating virtual system patterns, you will almost certainly be creating script packages.
Largely, the act of creating a script package is independent of IBM Workload Deployer. The appliance does not dictate a particular scripting language, so all you need to do is make sure you can invoke your logic in the operating system environment. Your script package may be a wsadmin script, shell script, Java program, Perl script, and on and on. After you create the actual contents of your script package, you will then load that asset into the IBM Workload Deployer catalog.
Once loaded into the catalog, you define several attributes of your script package, including the executable command, command arguments, variables, execution time, and more. The process for defining these attributes is trivial using the intuitive UI in IBM Workload Deployer, but I wanted to take a little time to remind you of a technique I recommend to all users defining script packages. You can actually package a JSON file within the script package that defines all of the script's attributes. The format of the file is simple, and I am including an example below:
The example above is one taken from a script package in our samples gallery, and it shows the basics of which you need to be aware. Notice that in the JSON file, you can provide a name, description, unzip location, executable command, command arguments, variables, and more. You only need to ensure that the name of this JSON file is cbscript.json and that you include it at the root of the script package archive. Once you have done that, you load the script package archive into the catalog, refresh the script package details, and voila -- all the attribute definitions appear!
You may ask why I recommend this since it could seem like an unnecessary step. My answer to that is that you have to define these attributes anyway, so you might as well capture it once in the file. Once you capture it once in the file, you can ensure that if the same script needs to be reloaded, or if you need to move it to another appliance, its definition will be exactly the same (and presumably correct). I use this approach for all of my work, and for all of the samples I contribute to our gallery, and it really saves me a lot of misplaced effort that can result from typos. If you are out there creating script packages, try adopting this approach. I'm pretty sure you will be happy you did!
In WebSphere CloudBurst, a script package is your vehicle to provide custom middleware configuration. This may mean installing applications, configuring application dependencies, or otherwise tuning the middleware layer. Script packages are essentially ZIP files that include some executable (shell script, wsadmin script, Java program, etc.), and optionally, artifacts that support the execution of the script. As was the intention, you can achieve just about anything you want with a script package. This allows you to be as flexible and creative as you need to be, but it can also leave you asking "Where do I start?" In this post, I want to take an in-depth look at constructing and using a script package in WebSphere CloudBurst.
Specifically, I want to create a script package that supplies configuration functionality for something I believe a fair number of you do: change the default ports used in WebSphere Application Server. To create this and deploy a pattern using the script package, I do the following:
Create a shell script that configures the desired ports
Add the new script as a WebSphere CloudBurst script package
Create a pattern with the new script package
Deploy the pattern and verify the result
First things first. I create the following shell script that configures the ports:
The script uses documented ANT commands included with the WebSphere Application Server to update the ports based on a starting port number. You will notice the script first sources the /etc/virtualimage.properties file. This file is automatically created by WebSphere CloudBurst on every virtual machine it starts. The file is a key/value file with basic information about the WebSphere cell such as the install root ($WAS_INSTALL_ROOT), the profile name ($PROFILE_NAME), host name ($HOSTNAME), and more. For a full list of the data that WebSphere CloudBurst includes in this file, check out this documentation.
In addition to utilizing the standard set of variables provided by WebSphere CloudBurst, my script above also makes use of the $STARTING_PORT variable. Obviously this variable is not in the standard set. In fact, I define the STARTING_POINT variable when I define my new script package in WebSphere CloudBurst.
First I zip up the shell script above and attach it to the new script package. Next, I tell WebSphere CloudBurst where to unzip the script package on the virtual machine, how to invoke the included script, and the name of any parameters to associate with the script. Once that is done I can use the script package in a new pattern.
For the sake of simplicity here, I create a new pattern by cloning an existing WebSphere Application Server single server pattern. I drag and drop the new Configure ports script package on the single part and end up with the pattern shown below.
Now I am ready to deploy the pattern by clicking the Deploy button. During the deployment process I configure each part in the pattern (in this there is only a single part). I supply configuration information like virtual memory allocation, WebSphere cell name, WebSphere node name, and password information. In addition, I also supply a value for the STARTING_PORT parameter that is part of the Configure ports script package included in the pattern. The value I supply here will get inserted into the /etc/virtualimage.properties file on the virtual machine, and the value's key will be STARTING_PORT.
Once the configuration information is supplied, I click OK on the configuration panel and deployment panel, and WebSphere CloudBurst goes about standing up my virtualized WebSphere cell and running my script to configure the ports for the server instance. When it is done, I login to the WebSphere Application Server administration console to verify my results. To do this, I navigate to the configuration for the single application server instance, and pull up its port definitions.
Based on the results I can see my customizations took effect. I successfully captured my own unique WebSphere environment (in this case with a custom port range) in the form of a pattern. This custom environment can be deployed as many times as I need, in an automated fashion, and I'm guaranteed consistent results each and every time.
I hope this gives you a better idea of what script packages are all about and how they can utilize both WebSphere CloudBurst and user-supplied data that exists in the /etc/virtualimage.properties file of each virtual machine. If you have any questions let me know. I'm on Twitter @damrhein, or you can leave a comment right here.