In my last post, I concentrated on the new enhancements to WebSphere CloudBurst 188.8.131.52. One of the major new additions was the introduction of Environment Profiles, and I promised a developerWorks article would be forthcoming. The article is now live along with a demo that showcases the capability of environment profiles.
As I mentioned in my last post, environment profiles center around giving you more customization capability during the pattern deployment process. In WebSphere CloudBurst, the pattern deployment process consists of the five main steps depicted below.
Traditionally, WebSphere CloudBurst controlled the entire deployment process, thus closing it off to the deployer. Environment profiles extend the customization reach of users to be able to effect steps 1-3 in the above diagram. Specifically, environment profiles give you the following control:
Control over the assignment of IP addresses and hostnames to pattern parts: Instead of having WebSphere CloudBurst automatically assign IP addresses, and thus hostnames, to virtual machines during deployment, you can explicitly set both values during the deployment process.
Ability to deploy single patterns to multiple cloud groups: Previously, when deploying a pattern you selected a single cloud group and WebSphere CloudBurst deployed all the parts in the pattern to machines within that cloud group. While this may be okay for many cases, other cases may require you to deploy some parts of the pattern to one group of machines while other parts map to a separate set of machines. Before environment profiles, you could accomplish this with multiple patterns. With environment profiles, you can accomplish it with a single pattern.
Ability to supply virtual machine naming standards: As part of deploying a pattern, WebSphere CloudBurst creates one to many virtual machines with distinct names. Environment profiles allow you to supply a naming standard that WebSphere CloudBurst will use when creating the machines as opposed to default naming schemes previously used.
It is important to note that the use of environment profiles is completely optional, and you can continue to use the traditional deployment process, thereby leaving WebSphere CloudBurst in control. That said, the introduction of environment profiles is a direct response to consistent user feedback we received regarding the need for more control during the deployment process. Based on my user conversations, these profiles address many of said needs in an easy to use, straightforward manner. We are, of course, eager to know what you think. As always, you can let me know right here, through email, or on Twitter (@damrhein).
I was at a customer meeting the other day, and someone asked me if they could query WebSphere CloudBurst for an inventory of all of their virtual system deployments. This person was of course aware that he could go to the web console and very quickly view all of the virtual systems. What he wanted though was something that he could run to generate a report that contained all of this information. For a purpose like this, harnessing the WebSphere CloudBurst CLI is exactly the way to go.
I thought I'd write a simple CLI script that provides an example of how you could do this.
from datetime import datetime
outFile.write("WebSphere CloudBurst Virtual System Inventory\n")
outFile.write("Total virtual systems: " + str(len(cloudburst.virtualsystems)))
def writeVSDetails(outFile, virtualSystem):
outFile.write("\tVirtual system name: " + virtualSystem.name)
outFile.write("\tCreated from pattern: " + virtualSystem.pattern.name)
outFile.write("\tVirtual system status: " + virtualSystem.currentstatus_text)
created = datetime.fromtimestamp(virtualSystem.created)
outFile.write("\tVirtual system creation date: " + created.strftime("%B %d, %Y %H:%M:%S"))
outFile.write("\tTotal virtual machines: " + str(len(virtualSystem.virtualmachines)))
def writeVMDetails(outFile, virtualMachine):
outFile.write("\t\tVirtual machine name: " + virtualMachine.name)
outFile.write("\t\tVirtual machine display name: " + virtualMachine.displayname)
outFile.write("\t\tCreated from image: " + virtualMachine.virtualimage.name)
outFile.write("\t\tVirtual machine hypervisor: " + virtualMachine.hypervisor.name + " | " + virtualMachine.hypervisor.address)
outFile.write("\t\tVirtual machine IP address: " + virtualMachine.ip.ipaddress)
outFileLoc = sys.argv
outFile = open(outFileLoc, 'w')
for virtualSystem in cloudburst.virtualsystems:
for virtualMachine in virtualSystem.virtualmachines:
As a result of invoking this script using the CLI's batch mode, content is written to the file location supplied by the caller.
WebSphere CloudBurst Virtual System Inventory
Total virtual systems: 3
Virtual system name: Single server
Created from pattern: WebSphere single server
Virtual system status: Started
Virtual system creation date: January 15, 2010 16:37:20
Total virtual machines: 1
Virtual machine name: Standalone 0
Virtual machine display name: Single server cbvm-110 default
Created from image: WebSphere Application Server 184.108.40.206
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
Virtual system name: Development WAS Cluster
Created from pattern: Custom WAS Cluster - Development
Virtual system status: Started
Virtual system creation date: January 18, 2010 14:08:46
Total virtual machines: 2
Virtual machine name: DMGR 0
Virtual machine display name: Development WAS Cluster cbvm-112 dmgr
Created from image: WebSphere Application Server 220.127.116.11
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
Virtual machine name: Custom Node 1
Virtual machine display name: Development WAS Cluster cbvm-111 custom
Created from image: WebSphere Application Server 18.104.22.168
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
Virtual system name: DB2 for development use
Created from pattern: DB2
Virtual system status: Started
Virtual system creation date: January 18, 2010 14:09:58
Total virtual machines: 1
Virtual machine name: DB2 Enterprise Server 32bit Trial 0
Virtual machine display name: DB2 for development use cbvm-113
Created from image: DB2 Enterprise 22.214.171.124 32-bit Trial
Virtual machine hypervisor: Ruth ESX | https://<hypervisor_host>/sdk
Virtual machine IP address: <ip_address>
I withheld IP addresses and host names above for obvious reasons, but if you ran the script against your environment you would see actual host name and IP address values. The script above is written once, and it can be subsequently run anytime you want an inventory of virtual systems running in your WebSphere CloudBurst cloud. There's other information available for virtual systems and virtual machines that I didn't show here, and you can retrieve it if necessary for your inventory report. In addition, I chose to print this information as regular text in a file supplied by the caller, but you might choose to generate the report in another format including XML, JSON, or anything else for that matter.
-- Dustin Amrhein
p.s. As with any sample code or script I provide here, the above is only a sample and offered as-is.
The reason I suggest the application proxy approach is twofold. First, it affords you the ability of having custom interactions with the REST API. For instance, you may insert logic into the server-side proxy code that returns only a subset of the JSON data contained in the response from the appliance. Alternatively, in an effort to reduce the chattiness on your client-side, you may join JSON data from multiple different REST requests to the appliance to fulfill a single client request. You may even decide to represent the data in an all together different format than JSON. All of these options and many more are available to you if you implement an application-based proxy to the REST API.
The second reason I suggest the application approach is that it is easier, and seemingly safer, to not deal with user passwords on the client-side. If you setup your application proxy, you can configure it to retrieve the appropriate password from a secure location (like an encoded file) based on information passed along in the request. This means the password information is only present in the request (in encoded form of course) from the application proxy to the WebSphere CloudBurst Appliance.
The good news about the application-based proxy approach is that it is simple to put in place. I composed one using the open source Apache Wink project. The Apache Wink project is an open source implementation of the JAX-RS specification (and then some), and it enables you to develop POJOs that are in turn exposed in a RESTful manner. In my case, I had a single resource POJO:
The Apache Wink runtime routes any HTTP GET request whose path is like /resources/* to the getResources method in the WCAResource class. This method passes along information taken from the query string (the host name of the target WebSphere CloudBurst Appliance and the requesting WebSphere CloudBurst username), as well as the HTTP path information and sends it on to the getResource method declared as follows:
The getResource method above uses the WebSphere CloudBurst host name and the request path to construct the URL for the corresponding WebSphere CloudBurst REST API call. Next, it constructs an Apache Wink Resource object and sends the REST request along to the WebSphere CloudBurst Appliance. How do we authenticate this request? We use the WebSphere CloudBurst username (sent as a query string parameter) to retrieve the appropriate encoded password information. Once we have that, we construct the necessary header for basic authorization over SSL.
The application-based proxy shown here is simply a pass-through. It does not manipulate the data returned from the WebSphere CloudBurst REST API, nor does it map a single client-side call to multiple REST requests. However, it would be simple enough to extend it to do any of those things. If you have any questions about the code here, please let me know. I'd be happy to share more of the code, or talk about how and where to extend it.
Application-centric cloud computing is the main thrust behind the new capabilities of IBM Workload Deployer v3.0. But what does that really mean? After all, application-centricity is really just a concept. Granted, it is an important concept, but it is fairly meaningless until it is put into action or implemented. IBM Workload Deployer does just that with its new Virtual Application Patterns (VAPs).
VAPs are the embodiment of the workload pattern approach I briefly discussed in an overview post a few weeks back. The idea with a VAP is to give the user an interface through which they can provide their application, specify dependencies, declare functional and non-functional requirements and then deploy. Of course application middleware is a part of the overall solution, but IBM Workload Deployer has the smarts to build, configure, and integrate the necessary infrastructure in order to support the user's application. This is completely hidden from the user, so they are liberated to focus on the application and its requirements.
If we scratch a bit further beneath the surface of a VAP, we see that these patterns contain three primary pieces. These primary pieces are components, links, and policies, and they are fundamental to understanding how virtual application patterns work. Let's start with the building blocks of VAPs, components. Put simply, components represent different resources and functionality profiles that make up your application environment. As an example, the IBM Workload Deployer Pattern for Web Applications is a VAP that contains components for an EAR file, WAR file, message queue, and any number of other components that are typical requirements for a web application. The components will certainly vary based on the workload type (i.e. the components included in a web application VAP would be different than those included in a batch application VAP), but they are the foundation of any VAP.
From the ground up, the next logical element we come to in the VAP is a link. A link is a way to declare a dependency or integration point between two components. As an example, consider a VAP with a WAR file component and a database component. You might draw a link between the WAR component and the database component to indicate that your web application uses or otherwise depends on the database. IBM Workload Deployer interprets this link, and takes it as a directive to configure the integration between the two components as a part of deployment. In this case, that may mean configuring a data source in the application's container. This is just a simple example, and an application may have any number of links between components.
Finally, we come to the policy element within the VAP. A policy is a way for a user to specify functional and non-functional requirements for their application environment. Users attach policies to the VAP, or to components in their VAP, and IBM Workload Deployer interprets and enforces those policies. In the context of a web application, one example of a policy could be a scaling policy. The scaling policy might indicate scaling requirements for the application that included minimum application instances, maximum application instances, and conditions that triggered scaling activities. IBM Workload Deployer would use the information in a scaling policy within a VAP to appropriately manage the deployed, running environment. Other examples of a scaling policy may include a JVM policy that provides configuration directives for the java virtual machines in your application environment or a logging policy that defines logging configuration options. In any case, the policy element allows VAP builders to influence the configuration and management of the application environment.
In the example VAP below you can see the use of components (Enterprise Application, Database, User Registry, Messaging Service), links (blue lines between components), and policies (Scaling Policy, JVM Policy):
In total, when I look at a VAP a particular word sticks out to me: declarative. VAPs really enable declarative, application-centric cloud computing. What do I mean? By declarative, I mean you are telling IBM Workload Deployer what you want, but not necessarily how you want it done. It is the job of IBM Workload Deployer to take care of the how. This shift in approach to application environments enables the potential for significant savings, and more importantly to me, lays the foundation for a more agile, flexible approach to deploying and managing application environments.
There will be more in the weeks and months to come on IBM Workload Deployer, so stay tuned. I also want to put a plug in for a new blog from Jason McGee. For those that do not know Jason, he is an IBM Distinguished Engineer, and the lead architect behind IBM Workload Deployer. Be sure to check out his blog for insights on this new offering, as well as for all things cloud.
I write a lot about WebSphere CloudBurst script packages. Typically, I write about what they are, how to create them, and even provide some samples from time to time. I find that most of the time I'm either writing or talking about script packages from the standpoint that they allow you to automate the delivery of customizations to environments you deploy with WebSphere CloudBurst. More specifically, I usually explain how you can include these script packages in your patterns to ensure that your custom scripts execute as part of every pattern deployment. The truth is, that is not the whole story. In fact, it's only 1/3 of the story.
In WebSphere CloudBurst, when you define a script package you also define its execution mode. The execution mode can be one of three values, and it indicates the invocation time for the script. The default value is at virtual system creation, and that tells WebSphere CloudBurst to automatically invoke the script as part of the deployment process. This seems to be the most commonly used execution mode, and in the original version of WebSphere CloudBurst it was the only available mode (which probably attributes to why I only usually tell 1/3 of the story here). As you may expect, there is a wide range of usage scenarios for this class of script packages including installing applications, activating monitoring agents, registering cells with an externally managed DNS server, and much more.
If you are like me (and many humans), you enjoy and actually expect symmetry. In that regard, it probably comes as no surprise that there is a script package execution mode called at virtual system deletion. As the name indicates, this class of script packages executes as an automatic part of the virtual system deletion process. When a user tells WebSphere CloudBurst to remove a virtual system, before it shuts down the machines in the system, it will run each script package marked to execute at virtual system deletion. Typical use cases for these scripts include removing information about the cell from externally managed DNS servers, freeing up connections with external systems, and other external 'clean up' activities.
So this leaves the final execution mode for script packages, the when I initiate it mode to be precise. This class of script packages executes when explicitly triggered by a user. In the virtual machine detail section for a deployed virtual system, you can see a list of user-initiated script packages for a given machine. There is a start button by each of the user-initiated script packages that allows you to invoke the script when, and as many times as you need to. While these script packages have many different use cases, the most common use case is to deploy application updates. Users build these application update scripts, attach them to a pattern, and invoke them whenever they want to deliver an updated application into their already deployed environment.
WebSphere CloudBurst script packages are one of the main vehicles for delivering your customizations to your cloud environments. The three execution modes mentioned above allow you to determine when the right time to deliver those customizations is.
I spent most of my time growing up doing two things, going to school and playing sports. I made many fond memories -- mostly from the latter :) -- and learned more than a few lessons over that time. Of all of those lessons, there was one in particular that stuck out in both the classroom and on the baseball diamond: Sometimes you have to get back to the basics.
In that vein, I think it is time to revisit the basics of WebSphere CloudBurst. In revisiting the basics, I am not talking about the technical basics of the appliance. Rather, I am talking about revisiting exactly why WebSphere CloudBurst exists in the first place. In other words, let's take a look at the problem domains WebSphere CloudBurst addresses, and let's discuss a little bit about how the appliance does so.
When many people think of cloud computing they immediately think of virtualization and virtual machines in particular. This is completely natural and not at all surprising. After all, one of the core underlying technologies necessary for cloud computing is virtualization. However, it is important not to confuse one element of cloud computing with the entire thing - and this can sometimes happen. Many people have begun to leverage virtual machines in their on premise environment and sometimes begin to call this their private cloud. While virtualization is a substantial step forward and help gets you started down the necessary path of standardization and automation that is essential in a cloud - it is not in and of itself "a cloud".
The National Institute of Standards and Technology has published its definition of cloud computing. This is a very complete and yet concise definition that includes not only the essential characteristics of a cloud solution but also the service models (IaaS, PaaS, SaaS) and deployment models (public, private, hybrid, community). It is a great way to get a perspective on cloud and can be useful when considering the solutions of various vendors.
Let me summarize the essential elements of cloud from this definition here:
broad network access
So, this is interesting. Not only is this much more than just virtualization - but virtualization isn't even mentioned in the list explicitly. Not to worry - virtualization is of course important and is included under the resource pooling topic. I would assert that virtualization is also necessary to facilitate the type of on-demand, self-service, elastically scaling resources that are leveraged in a cloud. What is crystal clear from this definition is that there is a lot more to a cloud solution than just virtual images and some hypervisor infrastructure upon which to run them. Somebody must provide the necessary on-demand/self-service capabilities, the network access to these services, the management of the resource pools, enabling true elasticity for running systems, measuring services and so forth. IBM Workload Deployer provides just such capabilities for the on-premise cloud allowing you to efficiently deploy patterns built for virtual systems and virtual applications with deep knowledge of the middleware that is being provisioned to optimize these solutions. Furthermore, Workload Deployer provides the complete lifecycle management from pattern creation, to deployment and provisioning, applying maintenance, resource and license management in the on-premise cloud, elastic scalability, and eventually returning resources to the on-premise cloud to be reused. Workload Deployer is a complete solution for not only server virtualization but of course for cloud computing.
However, virtualization doesn't have to stop with just virtual machines. It is a general principle that can be applied to more than just servers. At its core, virtualization is really about providing a level of abstraction between some real resources and the consumers of those resources. This is a natural fit when we think of server virtualization and virtual machines. However, there are also substantial benefits to be gained by adopting a similar abstraction between the middleware and the applications themselves - sometimes referred to as application virtualization.
By application virtualization I mean providing the capabilities to abstract the application from the underlying infrastructure such that it can be elastic, participate in health management policies, and provide agility across the pool of application infrastructure resources. This type of application virtualization is built into our Virtual Application pattern (hence the name) in Workload Deployer and surfaced in solutions via policies (such as scaling and routing), and high availability functions built into the Web Application pattern type. For Virtual Applications these features are fully integrated and optimized functions as are all elements of Virtual Applications. However, similar features have also been available for WebSphere Application Deployments in Virtual System patterns with a special extension.
WebSphere Virtual Enterprise provides application virtualization for traditional WebSphere ND solutions and this same feature is delivered for Virtual System pattern deployments of WebSphere Application Server by use of the Intelligent Management Pack. Leveraging the capabilities of Workload Deployer with Virtual Systems lets you gain the benefits of server virtualization and to reduce hardware, provide rapid and consistent deployment of entire systems, dynamically adjust resource consumption, and much more. Leveraging the capabilities of the Intelligent Management Pack provides the ability to manage service level agreements with elastic scaling and health management, lower operational costs, and provide for improved application management. These two solutions together provide a powerful combination to improve the management and resiliency of your enterprise applications.
If you would like to learn more about application virtualization using the Intelligent Management Pack in conjunction with Virtual System Patterns in IWD then please join Keith Smith and myself tomorrow for a webcast on this very topic. Keith is the lead architect on our WebSphere Virtual Enterprise and Intelligent Management Pack products and brings a wealth of experience in this space. So don't miss this opportunity - register here.
The ability to package custom maintenance packages and upload them as emergency fixes is perhaps a lesser known feature of WebSphere CloudBurst, but nevertheless something that's been around since the product's initial release. This is a powerful feature that allows you to build your own fix packages that you can then apply the same way you would use WebSphere CloudBurst to apply a PAK file or fixpack shipped by IBM.
Since IBM is delivering fixes and updates to all of the contents within WebSphere Application Server Hypervisor Edition virtual images (including the OS and IBM software components), you may wonder why you would even want to create your own maintenance packages. One reason would be if you switched out the SUSE Linux operating system shipped with the VMware ESX based images in favor of your own Red Hat operating system. In that case you would be responsible for maintenance to the operating system, and custom maintenance packages would be of interest to you. Another scenario where these custom maintenance packages come in handy would be if you created your own customized images that include non-shipped third-party software in addition to the software shipped in the images. If at some point you have the need to fix or update this software in a running virtual machine, custom maintenance packages provide you the vehicle with which to do just that.
What do these custom maintenance packages look like? In short, they are simply archives or ZIP files. The contents of the archive are largely decided by you, but there is one piece of metadata that is necessary if you want to use WebSphere CloudBurst to apply the maintenance. A file called service.xml is inserted into the root of the archive and tells WebSphere CloudBurst critical information about the custom fix archive. Here's an example of a service.xml file:
Most notably, this metadata tells WebSphere CloudBurst what module or script to invoke to apply the maintenance (Command, this executable is supplied by you), what image versions the fix is applicable to (ImagePrereqs), and the location of the working directory on the virtual machine (Location). In addition to the service.xml file and the executable, you can package up anything else, such as product binaries, which are needed to successfully apply the fix/upgrade/maintenance.
If you haven't noticed, this is an extremely flexible mechanism and can be used for just about anything. I should point out that you can only apply a given fix once per virtual machine, so it's not good for something that you want to run repeatedly against a given machine (check out user-initiated script packages instead). Also, there is a 512MB size limit on the archives. Keep these restrictions in mind when you are deciding how to use custom maintenance packages. If you are interested in learning a bit more about custom maintenance packages or other maintenance techniques, check out this article I co-authored along with Xiao Xing Liang from the IBM SOA Design Center in the China Development Lab.
A couple of weeks ago, I dropped by the Intel Developer Forum to present a session and listen in on a few others. As always in these types of shows, I learned quite a bit. Most strikingly though, I was reminded of something that is probably quite obvious to many of you: Consumer interest in cloud computing will not be letting up any time soon.
Based on this, and some of the other things I heard at the show, I decided to catch up with fellow IBMer Marc Haberkorn. Marc is an IBM Product Manager and is responsible for IBM Workload Deployer amongst other things. I asked him about IBM Workload Deployer, the competition, and cloud in general. Check out what Marc had to say below:
Me:IBM Workload Deployer is one among many of a growing wave of cloud management solutions. How do you differentiate the focus and business value of it versus the myriad of other solutions out there?
Marc: To sum it up, we offer a combination of depth and breadth. IWD delivers both workload aware management and general purpose management. Workload aware management differentiates IWD from its competition, as it can deliver more value for the set of products for which it has context. There is a set of actions that workload aware management tools can do that is normally left to the user by general purpose management tools. This list includes configuring a middleware server to know its hostname/IP address, configuring multiple middleware servers to know of one another, arranging clusters, applying maintenance, and handling elasticity. By handling more of these activities in the automated flow, there are fewer chances for manual errors and inconsistencies to enter a managed environment.
That said, without infinite resource or time, it’s impossible to deliver this context-aware management for everything under the sun. As such, in order to allow IWD to deliver differentiated value AND allow it to handle a customer's entire environment, we offer a mix of workload-aware management and general purpose management.
Me:VMware is a good example of a company active in the cloud space, and they seem to keep a consistent pace of new product delivery. What do you think of their product development focus?
Marc: I think VMware has built a very compelling set of capability in the virtualization space. I think the main difference between VMware's suite and IBM Workload Deployer is the perspective from which the environments are managed. VMware puts the administrator in the position of thinking about infrastructure from the ground up. The administrator is thinking about virtual images, hypervisors, and scripts. In IBM Workload Deployer, we think about things from the perspective of the app, because that's ultimately what the business cares about. By providing a declarative model through which an application can be instantiated and managed, we feel we deliver a deeper value proposition to clients, through workload-aware management.
Me:The 'one tool to do it all' approach is a popular, if not hard to achieve goal. What is your advice to users when it comes to choosing between breadth and depth for cloud management solutions?
Marc: The advantages of a "one tool to do it all" are many: less integration, more uniformity, less complexity. As such, customers will always prefer a single tool when possible. This is why IBM Workload Deployer has focused on not only providing differentiated, deeper value for common use cases but also providing a way to handle the "everything else." As such, my advice to users is not to choose between breadth and depth - use IBM Workload Deployer which offers both.
Me:To close, I'm curious to know where you think we are heading in the cloud market. What do you think users will be most readily adopting over the next one to two years? Where does the cloud industry need the most innovation?
Marc: I think most users are currently looking at the broad picture of cloud computing, and have been adopting primarily in the private cloud realm. There are several reasons for this. One reason is that many customers have a large set of hardware resources which amount to sunk cost that needs to be leveraged. Another reason is around data security concerns in off-premises clouds, and still another reason is around the human factor of comfort, which has taken time to develop around off-premise cloud models. However, businesses have become increasingly comfortable with various sources of outsourcing in recent years, especially in mission critical areas involving very sensitive data. Just look at IBM's Strategic Outsourcing business, which handles entire IT operations for many large businesses. I think that trend will (and really, has already begun to) continue in the area of cloud computing, and will lead to more public and ultimately hybrid cloud computing adoption. In order to get to hybrid cloud computing, I see much of the focus and innovation being associated with data security, workload portability (across private and public, in a seamless fashion), and license transferability between private and public. When this space reaches fruition, clients will be able to enjoy true elastic economics in a computing model that allows a mixture of owning and renting compute resources and software licenses.
If you've read anything I've written about WebSphere CloudBurst up to this point you know all about patterns. Using the appliance you can easily and quickly build, deploy, and manage these representations of your middleware application environments. Today, I want to focus in on the deployment piece in particular and take a look at how you can easily automate this process.
You can use the WebSphere CloudBurst web console to deploy patterns, and when doing so you can even schedule the deployment to happen at a later date. This scheduling capability certainly gets you on the road to an automated deployment process, but what if you want to take it one step further and eliminate the need for someone to login and manually move around the web console to schedule automated deployments? In this case, you can use either the CLI or the REST interface that WebSphere CloudBurst offers.
In this post I thought I'd take a look at using the CLI interface in order to set the stage for some nice automation around pattern deployment. It starts out with a properties file that provides details about my deployment. This includes the cloud to deploy to, the pattern to deploy, password information, and the time at which the virtual system should start.
SYSTEM_NAME_PREFIX=New App Development
TARGET_CLOUD=Default ESX group
TARGET_PATTERN=WebSphere single server
Imagine that the properties file above gets written as the result of some other action, such as the completion of your application's build process. With the properties file in place, and I'll point out that your properties file can and probably will be more robust than above, let's move on to the code that handles the deployment process based on the information in said file. First, we have a small amount of CLI code to retrieve and parse the input data (I omitted the straight-forward properties retrieval for space):
from datetime import datetime, timedelta
from java.util import Properties
from java.io import FileInputStream
// read in and retrieve properties using java.util.Properties API (i.e. props.getProperty('DEPLOYMENT_DATE'))
parsedParts = deploymentDate.split(" ")
systemName = systemName + "_" + deploymentDate
dateParts = parsedParts.split("/")
timeParts = parsedParts.split(":")
monthPart = int(dateParts)
dayPart = int(dateParts)
yearPart = int(dateParts)
hourPart = int(timeParts)
minutePart = int(timeParts)
Next is the code that actually schedules the pattern deployment:
First we get the desired deployment time and current time as datetime objects. After that, assuming the desired deployment time has not already elapsed, we calculate the difference between the desired deployment time and current time. This difference, in seconds, is then added to the result of the time.time() value to come up with a start time. After that is done, we simply retrieve the cloud that was indicated in the properties file, and then we call the runInCloud method for the pattern indicated. When calling the runInCloud method we supply the name of the virtual system that will be created, password information, and the start time we calculated earlier. As a result of this method call, a task will be generated in the target WebSphere CloudBurst Appliance and the virtual system will be started at the specified time. This will happen in an automated fashion with no human intervention required.
That's really all there is to automating the pattern deployment process using the CLI. In a more complete, end-to-end scenario you may envision the completion of one process, such as an application build process mentioned above, result in the writing of the properties file and in turn the call into the CLI to deploy a pattern. As always, feel free to send me any comments or questions.
It's really hard to complain about my work week right now. As I write this blog, I'm sitting in the Congress Center in Düsseldorf, Germany looking out over the Rhine River. As an aside, in Germany it is the Rhein River, and I have a historical connection to this body of water. My surname, Amrhein, translates (loosely) to 'on the Rhein'. It does not take an expert in genealogy to conclude that I have ancestors who at one time or another lived very close to this important German waterway.
Okay, putting the family tree aside for a minute, there is a good reason that I am in Düsseldorf this week. The city, and specifically the Congress Center, is playing host to the IBM European WebSphere Technical Conference. I am here presenting sessions that include a WebSphere CloudBurst overview, a WebSphere CloudBurst hands-on lab, and an up-close look at one of our internal team's use of the appliance. I have done each of these sessions once so far, and attendance was great, audience participation high, and feedback forthcoming. I am hearing and seeing the same thing in other sessions, which is of course, ideal for us presenters.
Now, to focus in on WebSphere CloudBurst for a bit, it seems that I am hearing a recurring question this week from the mostly European audience: "Why is WebSphere CloudBurst delivered as an appliance?" I am sure that I addressed this question in a previous blog post, but I believe it bears revisiting. There are various reasons I could give for the appliance form factor, but I like to distill all of that down into three major reasons: Consumability, Performance, and Security.
From a solution consumability perspective, nothing beats the appliance approach. WebSphere CloudBurst is an integrated hardware and software solution that delivers a specific set of function. You do not have to install software, procure and maintain storage for resources on the appliance (images, patterns, scripts, etc.), and maintain software components over time. You simply drop the appliance in to your data center, perform a one-time initialization, hook it up to the network, and you are ready to start leveraging WebSphere CloudBurst to build out your private cloud. While there is definitely work to setup the cloud infrastructure that WebSphere CloudBurst deploys environments to, we can completely eliminate a significant portion of solution implementation lead time by delivering everything you need in the appliance.
The performance benefits of an appliance approach are a natural result of building an integrated hardware and software stack. Design and development teams provide optimizations in both the hardware and software based on the fact that both the hardware and software have intimate knowledge of each other's design. In other words, this is not a 'least common denominator' tuning approach. Rather, the integrated design leads to enhanced performance for the specific set of functionality provided by WebSphere CloudBurst.
Finally, appliances enable us to deliver a very hardened, secure device. We provide private key encryption of every resource stored on the appliance. That private key is unique to each appliance and cannot be modified. In addition, the physical casing is tamper-resistant. If someone removes the casing, a 'Get Smart' style kill switch puts the appliance in a dormant state. You must send the appliance to IBM so we can reset it before further use, thus providing an additional layer of physical protection on top of the encryption. These security features, plus more, like a shield that prevents anyone from executing code on the appliance, come right out of the box and require no end-user configuration activity. In this way, you can simply focus on leveraging the user security and access controls provided by WebSphere CloudBurst.
If you had any questions on the rationale behind the appliance form factor of WebSphere CloudBurst, I hope this helps. I am off for now... back to the conference and the wonderful city of Düsseldorf.
I was very encouraged by the consistently positive response we got at IMPACT for our WebSphere CloudBurst and Rational Automation Framework for WebSphere (RAFW) integration. I believe there were many reasons for this response: accelerated time to value, decreased investment needs for activities that are not core to your business, lowered barrier of entry for provisioning and configuring WebSphere cells, and much more. While those are certainly all very real and valuable benefits, I also believe that quite a bit of interest in this integrated solution comes from the fact that it is applicable to a number of needs common to you, our WebSphere users.
With that in mind, let's look at some (not all) of the scenarios where WebSphere CloudBurst and RAFW integration can help you:
Create WebSphere CloudBurst patterns that include configuration without scripting: Users love our WebSphere CloudBurst patterns. They really see the value in codifying both the topology and configuration of their application infrastructure. However, some users do not have existing WebSphere configuration scripts and do not have the time and/or resource to invest in creating these scripts. They are looking for a solution that provides not only the provisioning of WebSphere environments but also the configuration of said environments (configuration beyond what our IBM Hypervisor Edition images encapsulate, specifically application-oriented configuration). RAFW provides this capability in the form of 500+ out-of-the-box configuration actions for WebSphere environments. This includes actions to create JDBC resources, create JMS queues, deploy applications, configure application containers, and much, much more. You can create WebSphere CloudBurst patterns that contain a special script package, which points back to a RAFW project containing a set of configuration actions. During deployment, WebSphere CloudBurst will provision your WebSphere environment and then cause the invocation of the specified RAFW project, which in turn runs a set of configuration actions against the provisioned environment. This means you can set up full-blown, ready-to-go application environments with absolutely no user-supplied scripting. In fact, I took this approach to setup a J2EE performance benchmark application, DayTrader 2.0, running on WebSphere Application Server. Those of you familiar with the application know this is not a trivial environment to stand up. Yet, I did it without having to personally write a single line of configuration scripting, and it was all ready to go in around thirty minutes.
Creating WebSphere CloudBurst patterns from existing environments: This comes up all the time. I go through a standard introduction to WebSphere CloudBurst, users see the value, love the patterns-based approach, and immediately want to know how they get their existing environments into the form of a pattern. RAFW, along with the special WebSphere CloudBurst script package, can make this a straightforward and hardened process. You use a capability in RAFW to import the configuration of an existing cell, thereby creating a RAFW environment for that configuration. You then create a WebSphere CloudBurst pattern with a topology congruent to your existing environment, attach the special script package I mentioned earlier, and you are done with the import! When you deploy this pattern, you simply specify the RAFW environment that you created earlier (the one that holds the configuration data for your existing environment) and a RAFW project that will apply the configuration data in that RAFW environment to the WebSphere environment provisioned by WebSphere CloudBurst. The creation of the WebSphere environment, as well as its configuration, happens in a completely automated fashion.
Configure, capture, reuse: There are many situations that may require you to make manual changes to a WebSphere cell after it has been deployed. For example, during performance testing for your application, you might discover that you need to tweak the number of available threads in the web container. As another example, for the first setup of a given application environment, you may want to quickly deploy the cell using WebSphere CloudBurst and then manually install and configure your applications to make sure everything is just right. In either case, it is likely that you want to capture the updated configuration and make sure that any future deployments use those updates. Again, WebSphere CloudBurst and RAFW makes this simple. First, you build a pattern that encapsulates your WebSphere topology (the types and quantity of nodes you want) and attach the special script package mentioned above. For the first deployment, you simply specify the name of the new RAFW environment you want to create. Once the system is up, you log into the WebSphere administration console, make your necessary customizations, and then you use RAFW to import that updated configuration thus updating the initially created RAFW environment. For subsequent deployments, you simply deploy the same pattern, specifying the same RAFW environment as well as a RAFW project, which RAFW automatically created for you during the first deployment. This project applies the configuration (the one you manually established and imported into RAFW) to the WebSphere environment setup by WebSphere CloudBurst.
Configure WebSphere environments across virtual and physical settings: It seems that in many cases our users manage the same WebSphere environment across both virtual and physical settings. For example, they may provision the application environment using WebSphere CloudBurst for everything from development to pre-production, and then for production provision that same environment to a set of physical servers. At least, they try to provision the same environment. In reality, it is tough to reproduce the exact same configuration once you break from the WebSphere CloudBurst patterns-based approach. However, if you stored the configuration of your WebSphere cell as a RAFW environment, you could apply that configuration data to a WebSphere cell regardless of whether it existed in the physical or virtual world. Once you move to physical, you do lose out on the fast provisioning, WebSphere intelligence, cloud management capabilities, and automated integration with RAFW that you get when using WebSphere CloudBurst, but if it is in your process to move to physical hardware at some point, reusing the same RAFW environment certainly eases the migration task.
I hope this sheds some light on some of the common issues WebSphere CloudBurst and the Rational Automation Framework for WebSphere can combine to solve really well. This is by no means an exhaustive list, but really meant to point out the broad application of the solution. If you want to see how it works, check out this video.
Among the major features of the new virtual application pattern in IBM Workload Deployer is the notion of elasticity. That is, as your application needs more resources, it gets them. When your application can meet its SLAs with fewer resources, the environment shrinks. With this kind of pattern, you enable elasticity by specifying a policy and defining the scaling trigger (i.e. CPU usage, application response times, database response times, etc.). What may have been a bit lost in some of these new announcements regarding IBM Workload Deployer is the fact that you can now leverage this core feature of cloud, elasticity, in your virtual system patterns.
If you have read this blog in the past, you probably already know that the Intelligent Management Pack is an option for virtual system patterns built using WebSphere Application Server Hypervisor Edition. When you enable the Intelligent Management Pack option, you are essentially building and deploying WebSphere Virtual Enterprise (WVE) environments. For those of you not familiar with WVE, the best way to describe it is that it provides you with application and application infrastructure virtualization capabilities. Of its many capabilities, one most germane to our discussion today is the ability for users to attach SLAs to applications and then have WVE automatically prioritize requests and manage resources in order to meet those SLAs. Inherent in this capability is the ability to dynamically start and stop application server processes (JVMs) as required. In other words, WVE provides JVM elasticity.
The fact that WVE provides JVM elasticity is nothing new. Further, IBM Workload Deployer started providing virtual machine (VM) elasticity in previous versions (when it was WebSphere CloudBurst). With this feature, you could add or remove VMs to an already deployed virtual system using dynamic virtual machine operations provided by the appliance. The catch was that the VM elasticity was a manual action and you could not link this elasticity to the same SLAs tied to your applications. Well, thanks to a new feature in WebSphere Virtual Enterprise and easy integration provided by the Intelligent Management Pack, this is no longer the case.
Starting in IBM Workload Deployer 3.0, you can take advantage of a new WVE feature called Elasticity Mode when using the Intelligent Management Pack. Elasticity mode is not unique to IBM Workload Deployer, but a concept new to the base WVE product. It allows one to define actions for how WVE should grow and shrink the set of nodes used by application server resources. Like the basic JVM elasticity capability in WVE, these node elasticity actions trigger based on SLAs tied to your applications. Consider the case that you are using elasticity mode and your application is not currently meetings its SLA. If WVE does not think it can start any more application server instances on the current set of nodes, it will grow the set of nodes per your elasticity configuration. Conversely, if WVE detects that it can meet SLAs with fewer nodes, it will shrink the resources per your elasticity configuration.
In IBM Workload Deployer, using elasticity mode becomes even easier. All you need to do is use the Intelligent Management Pack and enable the elasticity mode option in your virtual system patterns. When you do this, you get automatic integration between IBM Workload Deployer and the deployed WVE environment. What does that mean? It means that if WVE detects it needs more nodes, it will automatically call back into IBM Workload Deployer and request that the appliance provision a new VM that will serve as a node for application server processes. It also means that if WVE detects it could meet SLAs with fewer resources, it will call into IBM Workload Deployer and ask it to remove a node. All of this happens without any user scripting. All you have to do is enable this option in your patterns and configure SLAs appropriate for your applications.
To me, this exciting new feature brings out the best of elasticity capabilities in both IBM Workload Deployer and WebSphere Virtual Enterprise. The result is a single management plane that gives you both VM and JVM elasticity for your cloud-based application environments. Best of all, elasticity actions map directly to SLAs for your applications. After all, when it comes to cloud, it's the application that really matters!
A cloud is more than just coalesced water vapor. If it were then fog, mist and steam would all be considered a cloud. In truth, some definitions do say exactly that. However, we communicate most effectively when words have clear and distinct meanings. If I were to ask you to visualize a cloud, you would think of the puffs of grandeur in the sky. No matter what you think of in addition, that image would be invoked. Even if you are unsure of the context, that image would be amongst the most likely possibilities. Sky clouds, as envisioned, do require water vapor but they also require air, space, pressure and light to form that common image.
What's the point? The point is that the word, 'cloud', has a commonly understood meaning, regardless of the technical or scientific details that can make that specific meaning less exclusive. No one is served by making the definition more ambiguous. Similarly, the description and components of cloud computing should not be watered down to allow every conceivable enterprise feature or outcome.
Cloud computing is a way to maximize capacity and utilization and to minimize space, maintenance and to simplify governance. It offers these benefits by employing virtualization concepts and capitalizing on the emergence of patterns in enterprise topologies.
Virtualization is not a cloud solution, but a cloud solution will require virtualization in some form, whether it be cloning or full virtual images. Similarly, parallel processing on pooled resources is not a cloud but the principles of that are important to the conception of an effective cloud. However, a cloud also requires understanding of the enterprise, a clear picture of patterns and topologies and an efficient process for managing images as distinct entities. In other words, it's not just water vapor.
Cloud computing is the product of the evolution of networks and enterprises. It requires many things that have existed for years but only now have developed to the point where we can achieve the power and flexibility that cloud computing offers. Weighing down the grandeur of the cloud concept by overemphasizing some constituent part or by understating the importance of its management and governance serves no one except for the few trying to get a free ride in the sky.
To continue with the series of blog posts regarding WebSphere CloudBurst FAQs, I want to take a look at one aspect of the deployment process.
When you leverage WebSphere CloudBurst to push patterns (complete WebSphere Application Server configurations) into a private cloud, the appliance provides an advanced placement algorithm to determine exactly where the resulting WebSphere virtual systems will reside. It attempts to match the needs of the pattern to the correct set of hypervisors that have been defined. WebSphere CloudBurst considers things like storage, CPU, memory, and high availability characteristics when placing the pattern, and this is all done by the appliance without you having to intervene at all.
This is certainly nice in that it absolves you from having to make such placement decisions. Having said this though, you may be thinking of a question that comes up quite often:
If WebSphere CloudBurst controls the placement of the pattern, how can I make sure that certain deployments end up on certain servers (hypervisors)?
Considering what I just told you above, it may not seem that it's possible to control what machines end up hosting your virtual system since the appliance takes care of that placement for you. However, the organized use of WebSphere CloudBurst cloud groups allows you to take advantage of the intelligent placement provided by the appliance while retaining a level of control over which machines end up hosting particular deployments.
In WebSphere CloudBurst all patterns are deployed to cloud groups. Cloud groups are a collection of hypervisors that have been defined within the appliance. The basic deployment mapping is depicted in the image below:
As seen above, you can create a cloud group for any purpose (dev, test, QA, production, etc.), including any hypervisors that you desire as long as a given hypervisor only belongs to a single cloud group. When you are ready to deploy a pattern, you simply select the cloud group you want to deploy to:
By selecting a cloud group for deployment, you are implicitly selecting the physical machines that will host your deployment. The cloud group could consist of anywhere from one to N hypervisors, so you are afforded the ability to restrict the location of your virtual systems as necessary.
I hope this helped explain a little bit about cloud groups in WebSphere CloudBurst. If you're looking for more information about WebSphere CloudBurst cloud groups, I'd also suggest you watch this video on our YouTube channel.
In WebSphere CloudBurst, a script package is your vehicle to provide custom middleware configuration. This may mean installing applications, configuring application dependencies, or otherwise tuning the middleware layer. Script packages are essentially ZIP files that include some executable (shell script, wsadmin script, Java program, etc.), and optionally, artifacts that support the execution of the script. As was the intention, you can achieve just about anything you want with a script package. This allows you to be as flexible and creative as you need to be, but it can also leave you asking "Where do I start?" In this post, I want to take an in-depth look at constructing and using a script package in WebSphere CloudBurst.
Specifically, I want to create a script package that supplies configuration functionality for something I believe a fair number of you do: change the default ports used in WebSphere Application Server. To create this and deploy a pattern using the script package, I do the following:
Create a shell script that configures the desired ports
Add the new script as a WebSphere CloudBurst script package
Create a pattern with the new script package
Deploy the pattern and verify the result
First things first. I create the following shell script that configures the ports:
The script uses documented ANT commands included with the WebSphere Application Server to update the ports based on a starting port number. You will notice the script first sources the /etc/virtualimage.properties file. This file is automatically created by WebSphere CloudBurst on every virtual machine it starts. The file is a key/value file with basic information about the WebSphere cell such as the install root ($WAS_INSTALL_ROOT), the profile name ($PROFILE_NAME), host name ($HOSTNAME), and more. For a full list of the data that WebSphere CloudBurst includes in this file, check out this documentation.
In addition to utilizing the standard set of variables provided by WebSphere CloudBurst, my script above also makes use of the $STARTING_PORT variable. Obviously this variable is not in the standard set. In fact, I define the STARTING_POINT variable when I define my new script package in WebSphere CloudBurst.
First I zip up the shell script above and attach it to the new script package. Next, I tell WebSphere CloudBurst where to unzip the script package on the virtual machine, how to invoke the included script, and the name of any parameters to associate with the script. Once that is done I can use the script package in a new pattern.
For the sake of simplicity here, I create a new pattern by cloning an existing WebSphere Application Server single server pattern. I drag and drop the new Configure ports script package on the single part and end up with the pattern shown below.
Now I am ready to deploy the pattern by clicking the Deploy button. During the deployment process I configure each part in the pattern (in this there is only a single part). I supply configuration information like virtual memory allocation, WebSphere cell name, WebSphere node name, and password information. In addition, I also supply a value for the STARTING_PORT parameter that is part of the Configure ports script package included in the pattern. The value I supply here will get inserted into the /etc/virtualimage.properties file on the virtual machine, and the value's key will be STARTING_PORT.
Once the configuration information is supplied, I click OK on the configuration panel and deployment panel, and WebSphere CloudBurst goes about standing up my virtualized WebSphere cell and running my script to configure the ports for the server instance. When it is done, I login to the WebSphere Application Server administration console to verify my results. To do this, I navigate to the configuration for the single application server instance, and pull up its port definitions.
Based on the results I can see my customizations took effect. I successfully captured my own unique WebSphere environment (in this case with a custom port range) in the form of a pattern. This custom environment can be deployed as many times as I need, in an automated fashion, and I'm guaranteed consistent results each and every time.
I hope this gives you a better idea of what script packages are all about and how they can utilize both WebSphere CloudBurst and user-supplied data that exists in the /etc/virtualimage.properties file of each virtual machine. If you have any questions let me know. I'm on Twitter @damrhein, or you can leave a comment right here.
IBM Impact 2012 was just last week with a theme of "Change the Game" ... and I'm still reveling in all of the excitement and energy that goes into conferences such as this. I was fortunate to get a last minute spot to attend the conference and help out at the Solution Center where I had the chance to speak to a lot of customers and other IBMers interested in cloud computing. Among the many things that stood out - there is certainly a lot of interest in cloud computing and patterns of expertise - it also seems that folks are ready to get some first hand experience with these patterns. There's plenty of opportunity for that!
It's about the time of year when we all look back and try to determine exactly how we spent the past twelve months. Whether we do it because we have to as part of year-end job reviews or because we like to take stock in what we've done and figure out where to improve next year, it's a time for reflection and recall. For me, this exercise made me take a look at various things we have done to deliver WebSphere CloudBurst technical collateral (articles, demos, blogs, etc.) in 2009.
For all practical purposes, our mission and efforts for such technical collateral for WebSphere CloudBurst started when it was announced at Impact in May of this year. Though there was certainly some preparatory work being done on this front, there was nothing we could really push to the public until after the announcement, and in some cases even after the appliance's release in June. Given that most of the content was produced over a six month stretch, I really think we put forth a strong effort, and I hope that this technical material has helped to both raise awareness of and educate users on the WebSphere CloudBurst Appliance.
Seeing as I already went back and rounded up this content, I thought I'd provide you a centralized look at the information. To start, I accounted for the articles that we published to the IBM developerWorks site over the six month stretch. All together I counted 8 articles and a special column entry:
As you can see the articles cover quite a bit of content and range from general level overview articles to technical in-depth "how-to" style articles. In general they seem to have been received well with over 26,000 views to this point. Our goal is to keep the pace up for 2010, and we already have a few articles on our plate for early in the new year (including an overview of what's new in WebSphere CloudBurst 1.1).
Another main medium we utilized to spread the word about WebSphere CloudBurst was YouTube. On our YouTube channel at http://youtube.com/websphereclouds, we currently have 17 different videos that demonstrate how to use certain features of the WebSphere CloudBurst Appliance. Though I think each demo provides value depending on exactly what a viewer is looking for, 3 of them really stick out for me.
Check out our videos if you get a chance. We've made an effort to keep them as short as possible while still providing value to viewers.
We have some WebSphere CloudBurst content spread around other places as well including this blog and my personal blog. Over the next few weeks we'll be taking a look at what worked and didn't work with respect to getting information out to the public. Of course at any time we very much appreciate your feedback on how you like to see content delivered because you are our target audience! If you have a comment, idea, or suggestion, leave a comment on the blog or send me a tweet to @WebSphereClouds.
When it comes to managing users and user groups within WebSphere CloudBurst, you can choose to manage all aspects of those resources within the appliance. Mainly this means that you can define and store user information (including login passwords) within the appliance, and you can define and maintain user groups and their associated membership list on the appliance. While you can do this and be sure that your information is extremely secure, you may instead want to integrate with an existing LDAP server that has some of this user and user group data. WebSphere CloudBurst certainly allows you to integrate with LDAP servers, but what does that mean for you?
For starters, when you integrate WebSphere CloudBurst with an LDAP server and enable the LDAP authentication feature, you no longer specify password information when defining users of the appliance. When users login, the password they specify will be authenticated against information stored in the LDAP server. Naturally, if you add a new WebSphere CloudBurst user with LDAP authentication enabled, that user must be defined in the LDAP server. Otherwise, WebSphere CloudBurst will prevent you from adding the user because it has no way to authenticate that person.
From a user groups standpoint, integrating with LDAP means you can no longer modify user group membership. User membership in groups is determined by information in the LDAP server. As a result, the same rule concerning adding new users applies when adding new user groups: You cannot define new user groups that do not exist in the LDAP server.
If you want to take a look at what LDAP integration looks like with WebSphere CloudBurst, I put together a short video. Let me know what you think.
A while back I co-authored an article along with Chris Ahl from Tivoli and Ken Klingensmith from WebSphere Technical Sales about the customization of virtual images in WebSphere CloudBurst. In the article we approached image customization as a means to enable IBM Tivoli Monitoring for the operating system within virtual machines dispensed by WebSphere CloudBurst. Today I posted a short demonstration that discusses and shows this particular integration scenario. If you are interested, but haven't had time to read the article, you may want to watch the video first as it should give you a good overview of the process and results.
Talk of Tivoli reminds me that IBM Pulse 2010 is just around the corner. I'll be going to discuss WebSphere CloudBurst and how it can be paired with software from IBM Tivoli for high-value integration scenarios. In the session I'll be talking about the Tivoli Monitoring integration as well as other key points such as our integration with Tivoli Service Automation Manager, IBM CloudBurst, and more. The best part about the session is that I will be co-presenting alongside a WebSphere CloudBurst customer that will dole out practical advice for using WebSphere CloudBurst within the enterprise. Join us on Tuesday February 23rd from 3:30 - 4:30 in Conference Center 306.
Remember, any time you have questions about WebSphere CloudBurst please pass them along. You can leave comments on this blog, or you can reach me at my new Twitter location @damrhein.
When it comes to IBM Workload Deployer, I have no illusions concerning our competitors. They are out there, and they are constantly on the attack. Their dubious claims aside, I know this because I still get asked quite frequently to explain the benefits of IBM Workload Deployer versus some other general purpose cloud provisioning and management solution. So, while I have done that many times in various forums, I figured it was time to address the subject here on the blog.
When comparing IBM Workload Deployer to the other available solutions, I honestly feel comfortable saying we have no direct competition. I know you believe me to be biased, and rightly so, but let me explain why I think the competition is much more perception than reality. To do this, I want to focus on the patterns-based approach that IBM Workload Deployer takes to cloud provisioning and management.
Let's start with virtual system patterns in IBM Workload Deployer. Virtual system patterns allow you to build and deploy completely configured and integrated middleware environments as a single unit. These patterns build on top of our special IBM Hypervisor Edition images that bottle up the installation and quite a bit of the configuration of the underlying middleware products. Further, when using virtual system patterns, IBM Workload Deployer manages and automates the orchestration of the integration tasks that need to happen to setup a meaningful middleware environment. For instance, when deploying WebSphere Application Server you do not need to do anything on your end to deploy a clustered, highly available environment. When deploying WebSphere Process Server in this manner, you do not need to take any administrative actions to produce a golden topology. You just deploy patterns and the images, patterns, and appliance take care of the rest. Of course, you can add your own customizations and tweaks in the pattern, but we take care of the common administrative actions that would otherwise require your care.
I am not sure of a better way to say it, so I will be blunt: When deploying products delivered in IBM Hypervisor Edition form, no other solution compares to the virtual system pattern capability offered by IBM Workload Deployer. It is not even close. Can you provision products like WebSphere Application Server or WebSphere Portal using other cloud provisioning tools? Sure, but you should be aware that you will be writing and maintaining your own installation, configuration, and integration scripts. It is also likely that you will end up developing a custom interface through which deployers request your services (something not necessary when using the slick IBM Workload Deployer UI). All of this takes time, resource, and money. More importantly, this is not differentiating work and distracts from the real end goal: serving up applications. IBM Workload Deployer can deliver this operational capability right out of the box, and it can do so in a way that costs less than custom developed and maintained solutions.
When considering IBM Workload Deployer versus the competition, it is also important to consider the new virtual application pattern capability delivered in version 3.0. The virtual application pattern capability is a testament to IBM's thought leadership in and commitment to cloud computing for middleware application environments. Virtual application patterns take a bold step forward in raising the level of abstraction beyond the middleware environment and up to the most important resource in enterprise environments: the application. With a virtual application pattern, you simply provide your application and specify both functional and non-functional requirements for that application. When ready, you deploy that pattern, and IBM Workload Deployer sets up the necessary middleware infrastructure and deploys the provided application. Moreover, the appliance will monitor and autonomically manage the environment (i.e. scale it up and down) based on the policies you specify. Quite simply, this is a deployment and management capability our competition cannot match.
There is more to consider than just patterns though. The appliance makes it really simple to apply maintenance and upgrades to environments running in your cloud. It can autonomically manage your deployed environments (through policies in virtual application patterns and the Intelligent Management Pack for virtual system patterns), and it effectively abstracts the underlying infrastructure of your cloud environment. This abstraction is the reason IBM Workload Deployer can deploy your environments to PowerVM, zVM, and VMware environments. It also makes it easy to deploy the same environment to multiple different underlying platforms, thus accommodating typical platform changes that happen as an application moves from development to production. The best part of all is that the deployer’s experience is the same regardless of the underlying infrastructure since the appliance hides any platform idiosyncrasies.
The bottom line is that the appliance is purpose built to deploy and manage middleware and middleware application environments in a cloud, and as such, delivers immense out-of-the-box and ongoing value in this context. I should also point out that the design of the appliance acknowledges its purposeful nature. The CLI and REST API interfaces allow you to integrate the appliance into the operations of those general purpose provisioning solutions. In this way, IBM Workload Deployer acts as a middleware accelerator for your cloud computing efforts. This means that if you do have a general purpose solution, IBM Workload Deployer can still provide considerable value and let you avoid developing a considerable subsystem dedicated to deployment and management of middleware in the cloud. We believe in this type of integration, and have in fact built it into our own IBM solutions.
I could go on and on differentiating IBM Workload Deployer from the competition, but I hope my comments above give you a good context on why I think the appliance is in a league of its own. Of course, I always appreciate comments and feedback, so don't be shy!
May is almost here and that means that IBM IMPACT is right around the corner. Just like years past, IMPACT 2010 will be a great chance to get valuable education and insight into IBM WebSphere software and software from across the IBM software family. If you want to hear how IBM software is leading the march toward a smarter planet, register now.
IMPACT 2010 will be a great chance to hear the WebSphere cloud computing story. There will be multiple sessions on the WebSphere CloudBurst Appliance. These include customer-led sessions, internal adoption stories, overviews, and much more. I'll be there running a hands-on lab and delivering a session that discusses integration between WebSphere CloudBurst and IBM Rational tools. Of course, there is more to WebSphere and cloud computing than WebSphere CloudBurst. We have several other sessions that will detail all of IBM WebSphere's work in the cloud.
If you are interested, I put together a short video discussing some of the sessions on tap for WebSphere and cloud computing at IMPACT 2010. I'd also encourage you to check out the social media site for IBM IMPACT 2010. On that site, you will find tweets, videos, and blogs about the conference. Don't forget to sign up, and I hope to see you in Las Vegas!
-- Dustin Amrhein
If you frequently find yourself setting up and tearing down application environments that run on offerings from the WebSphere portfolio (like WebSphere Application Server or WebSphere Process Server), I have little doubt that you see the benefit of WebSphere CloudBurst. The appliance allows you to setup these environments with unprecedented speed and extreme simplicity. In fact, WebSphere CloudBurst makes it so simple and fast to setup these environments, it would be surprising if you did not spin up more WebSphere application environments with WebSphere CloudBurst than you did before your adoption of the appliance. Soon, you will find yourself faced with another challenge: that of managing and governing an increasingly growing ecosystem of your application environments.
From the beginning, WebSphere CloudBurst focused on the complete lifecycle for WebSphere application environments in an on-premise cloud. Therefore, in addition to easily creating and deploying these environments, the appliance delivers many features that help you manage and govern the dispensed virtual systems. This includes capabilities such as usage monitoring, fix and upgrade application, and virtual system state management. In the recently announced WebSphere CloudBurst 2.0, management capabilities go a step further, and now you can manage software license usage for your on-premise cloud.
What does it mean to be able to manage your software licenses? Well, in the new version of the appliance (firmware released planned for June 18th), as you dispense environments, WebSphere CloudBurst will keep track of the PVUs you are consuming for the particular IBM software you are instantiating. In doing this, it accounts for the physical machine architecture on which the supporting hypervisor sits, and it takes into account the IBM subcapacity/virtualization licensing policy. This means you can get an accurate view of your PVU usage at any point, and the appliance can produce a highwater mark report for any product over a date period you specify. This is license counting made easy!
In addition to simply tracking your PVU usage, you can optionally configure enforcement behavior. Enforcement behavior tells the appliance what to do when you exceed your PVU threshold for a particular product. You have three basic options: Ignore, Warn, Enforce. In Ignore mode, nothing happens when you exceed your PVU entitlement for a given product. Deployments that use those products continue to deploy as usual. In Warn mode, deployments for products for which you have exceeded your PVU entitlement continue as usual, but appliance administrators receive an email warning them of the situation. Lastly, in Enforce mode deployments that will put you over your PVU threshold for a given product simply fail. This prevents you or deployers using your appliance from overstepping your entitlement.
The software license management features in WebSphere CloudBurst 2.0 really add to the overall management capabilities of the appliance. I want to be sure to reiterate that the configuration of enforcement behavior, specifically the Warn and Enforce modes, is optional. It is not required from IBM. The software license management capabilities delivered in WebSphere CloudBurst 2.0 are purely meant to enhance your capability to manage and govern environments in your on-premise cloud. If you are interested in seeing this in action, check out this short video.
Over time, many of our users learn to effectively leverage WebSphere CloudBurst user roles and fine-grained access controls to map activities and responsibilities in the appliance to the appropriate people and teams within their organization. Using these controls, they are able to define actions that a user or group can take, and they can define the set of resources on which they can take those actions. It is efficient, flexible, and an absolute necessity in many enterprise scenarios.
In some cases though, I talk with users that want a little more control, or probably better put, governance over the actions a user can take within a given role. Most often, this need arises when the discussion of pattern authoring comes up. If you want a user in WebSphere CloudBurst to be able to create patterns, you simply give them the Create new patterns permission. Once you give them that permission, the user can create patterns using both virtual image parts and script packages in the catalog. For many of the users I talk with, this approach suits their needs.
However, in some scenarios administrators want a little more insight and control over how the pattern authors build their patterns. Specifically, they want to ensure that patterns contain only approved virtual image parts and script packages. While you can certainly use the fine-grained access controls of the appliance to expose only the 'approved' virtual image parts and script packages, that alone may not be enough. After all, the definition of what is 'approved' may be different when building a pattern for testing purposes versus one built for production purposes. If the same pattern author builds both of those patterns, fine-grained access controls do not help as much. So, what can you do?
Have I ever told you how much I love the WebSphere CloudBurst CLI? It's powerful, easy to use, and a great automation enabler. It is also the perfect tool for our problem above. If you are looking to enforce certain constraints on WebSphere CloudBurst patterns, I strongly recommend using the CLI as a governance tool.
To provide a concrete example of what I mean, let's take a look at a generic pattern checking script I am working on now (I hope to have this in the samples gallery soon). Consider the case that I want to check that all of my test patterns for a specific application environment contain 1 deployment manager and between 1 and 3 custom nodes. In addition, I want to make sure that the parts for these nodes come from an approved virtual image, and I want to verify that the deployment manager contains the correct application installation script package. With the script I am currently writing, you would start by encapsulating this information in a properties file.
PatternAssertion_1=Customer Processing Test Environments
PatternAssertion_1_Requirements=Deployment manager:1:415:Install customer process app;Custom nodes:1-3:415
In the above, the PatternAssertion_1 key provides a name for the pattern verification assertion. The PatternAssertion_1_Requirements key provides the requirements for the pattern. The above requirements indicate that for a pattern to meet the assertion, it must contain 1 deployment manager part from the virtual image with reference number 415. In addition, the deployment manager must contain a script named Install customer process app. A valid pattern must also contain 1 to 3 custom node parts, also based on the virtual image with reference number 415. When done defining my requirements in the properties file, I simply invoke a script and pass in the file. As a result, I get information about which patterns satisfy or do not satisfy the assertions. For example:
The Customer Process Application pattern satisfied the requirements of the Customer Processing Test Environments assertion.
OR The Customer Process Application pattern did not satisfy the requirements of the Customer Processing Test Environments assertion
due to the following reason: The pattern is required to have a minimum of 1 and a maximum of 3 Custom node part(s), but it had 4.
As I said, I hope to have this sample script posted to the samples gallery soon. I am going through some final revisions and enhancements that I hope make it better and more generally applicable. In the meantime, I wanted to point out that pattern governance is indeed doable, and in fact not very hard to achieve with the CLI. I will be sure to post an update when the sample script is ready. In the meantime, let me know if you have any questions or comments.
As far as easy to use, intuitive web interfaces go, I think WebSphere CloudBurst stacks up well against any competition. I think this is one of the main reasons why, for those with at least basic familiarity with WebSphere, the product has such a small learning curve. All that said, web interfaces are not ideal for all types of use cases. Namely, it would be nearly impossible to use WebSphere CloudBurst as part of automated processes if all it had were a web interface.
The need to automate the use of WebSphere CloudBurst or to use it within other automated processes is the reason behind the command line interface. The WebSphere CloudBurst CLI provides a Jython-based API with which you can leverage the same capabilities that are available in the web console. You simply unzip the CLI tools on any machine, and you can remotely interface with an appliance of your choosing. Sounds useful, right? Most agree that it does, but the question that comes up usually is, "How do I get started?"
I would suggest that anyone getting started with the CLI take a look at the overview and premise behind the API. Beyond this, it largely depends on how familiar you are with Jython scripting. If you are a WebSphere administrator that does a fair amount of wsadmin scripting, then you are probably all set. Otherwise, I suggest you spend some time in the interactive shell mode of the CLI, and in particular, I suggest you leverage the Wizard object provided by the CLI.
The Wizard object enables prompt-based creation of resources when using the WebSphere CloudBurst CLI. Take for example the following code snippet where I create a new virtual system:
>>> w = cloudburst.wizard()
Enter ?? for help using the wizard.
name: Single WebSphere Server
pattern (* to select from list): *
1. WebSphere single server
2. WebSphere cluster
3. WebSphere cluster (development)
4. WebSphere cluster (large topology)
5. WebSphere single server with sample
pattern (* to select from list): 1
cloud (* to select from list): *
1. Default ESX group
cloud (* to select from list): 1
Part properties and script parameters still requiring values:
value for 'part-1.ConfigPWD_ROOT.password': passw0rd
Part properties and script parameters still requiring values:
value for 'part-1.ConfigPWD_USER.password': passw0rd
number/key/-/*/?/??/!/enter to proceed:
"acl": (nested object),
"created": Jul 13, 2010 8:14:05 AM,
"maintenances": (nested object),
"name": "Single WebSphere Server",
"owner": (nested object),
"pattern": (nested object),
"snapshots": (nested object),
"updated": Jul 13, 2010 8:14:08 AM,
"virtualmachines": (nested object)
That's how easy the Wizard object makes using the CLI. Now, you might say that this is all well and good, but since the Wizard object implies prompt-based interaction, it is not helpful in terms of automating a process. That may be true for the first time you use the Wizard to create a particular resource, but if you take advantage of a handy method on the object it can enable automation going forward. The method I am talking about is the toDict method. By calling this method after the creation of a resource with the Wizard object, you get the dict object created from the information you entered via the prompts.
I truncated the output above for space, but the toDict method gives me the input data used to create the virtual system resource. This is really helpful going forward, as it gives me the exact input format to use to create my virtual system resources. I do not have to rely on the Wizard object any longer, and instead I can create virtual system resources without requiring direct user interaction because I know the input data WebSphere CloudBurst expects.
If you are just getting started with Jython-based scripting and the WebSphere CloudBurst CLI, I strongly suggest you use the Wizard object as a fast on-ramp. To move your automation work forward, make use of the toDict method. It will make writing completely automated WebSphere CloudBurst scripts much simpler. Good luck!
Script packages are an integral part of virtual system patterns in IBM Workload Deployer. By attaching script packages to your patterns, you provide customizations particular to your unique cloud-based middleware environments. Customizations provided by script packages might include installing applications, creating application resources, integrating with external enterprise systems, and much more. The bottom line is, if you are creating virtual system patterns, you will almost certainly be creating script packages.
Largely, the act of creating a script package is independent of IBM Workload Deployer. The appliance does not dictate a particular scripting language, so all you need to do is make sure you can invoke your logic in the operating system environment. Your script package may be a wsadmin script, shell script, Java program, Perl script, and on and on. After you create the actual contents of your script package, you will then load that asset into the IBM Workload Deployer catalog.
Once loaded into the catalog, you define several attributes of your script package, including the executable command, command arguments, variables, execution time, and more. The process for defining these attributes is trivial using the intuitive UI in IBM Workload Deployer, but I wanted to take a little time to remind you of a technique I recommend to all users defining script packages. You can actually package a JSON file within the script package that defines all of the script's attributes. The format of the file is simple, and I am including an example below:
The example above is one taken from a script package in our samples gallery, and it shows the basics of which you need to be aware. Notice that in the JSON file, you can provide a name, description, unzip location, executable command, command arguments, variables, and more. You only need to ensure that the name of this JSON file is cbscript.json and that you include it at the root of the script package archive. Once you have done that, you load the script package archive into the catalog, refresh the script package details, and voila -- all the attribute definitions appear!
You may ask why I recommend this since it could seem like an unnecessary step. My answer to that is that you have to define these attributes anyway, so you might as well capture it once in the file. Once you capture it once in the file, you can ensure that if the same script needs to be reloaded, or if you need to move it to another appliance, its definition will be exactly the same (and presumably correct). I use this approach for all of my work, and for all of the samples I contribute to our gallery, and it really saves me a lot of misplaced effort that can result from typos. If you are out there creating script packages, try adopting this approach. I'm pretty sure you will be happy you did!
One of the things I haven't written about much here is how the WebSphere CloudBurst Appliance integrates with other IBM software solutions. One of those interesting integration scenarios, and one I think is particularly useful for developers, involves Rational Build Forge.
Very simply put, Rational Build Forge is an adaptive execution framework that allows users to define completely automated workflows for just about any purpose. These workflows are represented as projects that contain a discrete number of steps. When looking at Rational Build Forge through the software assembly prism, the offering allows users to fully automate and govern the process of building, assembling, and delivering software into an application environment.
Now, on to the integration of WebSphere CloudBurst and Rational Build Forge. Users can build custom patterns in WebSphere CloudBurst that include a special script package (which I'll eventually provide a link to from here). This script package provides the glue between the deployment process in WebSphere CloudBurst and Rational Build Forge. When deploying a WebSphere CloudBurst pattern that contains this script package, users provide the name of a Rational Build Forge project as well as information about the Rational Build Forge server on which the project is defined.
Once the necessary information is supplied, the deployment process gets underway. Toward the end of the deployment, like all other scripts included in patterns, the special Rational Build Forge script is invoked. This results in the project specified during deployment being executed on the virtual machine created by WebSphere CloudBurst.
Because the Rational Build Forge project executes on a virtual machine setup by WebSphere CloudBurst, the individual steps of the project can very easily access the WebSphere Application Server environment. Thus, the Rational Build Forge project could very easily contain steps to build, package, and deploy an application into the WebSphere Application Server cell. The result is a fully automated process that includes everything from standing up the application environment to delivering applications into that environment.
I put together a short demonstration of this integration, and you can take a look at it here. As always, please let us know if you have any questions or comments. Your feedback is much appreciated!
As I have mentioned before, IBM Workload Deployer v3.0 introduces choices in pattern-based deployment models. One of those models, virtual system patterns, is a carry over from the WebSphere CloudBurst Appliance. When you use virtual system patterns in IBM Workload Deployer, you can take advantage of all of the techniques you put to use in WebSphere CloudBurst. This is certainly good news for current WebSphere CloudBurst users, but it goes a bit further. Instead of simply maintaining the status quo with virtual system patterns, which would have been reasonable considering the introduction of virtual application patterns, we chose to continue to expand on your customization options for this pattern deployment model. In particular, I want to discuss three new features in IBM Workload Deployer that may help you to better construct and manage virtual system patterns.
The first new feature is one that I have been eagerly awaiting. In the new version of the appliance, we provide you with the ability to specify part and script package ordering in your pattern. This means that, within the virtual system pattern editor, you can tell IBM Workload Deployer in which order to start the virtual machines in your pattern, and you can specify in which order to invoke the script packages within the pattern during deployment. This eliminates the need for special script invocation orchestration logic in your pattern (I had customers resorting to a semaphore like approach using a shared file system), and it allows you to be more declarative about the virtual machine bring-up process. There are constraints, specifically with the part ordering. Some images will impose an implied part start-up order that you cannot change. For instance, deployment manager parts in the WebSphere Application Server Hypervisor Edition image must start before custom node parts. The good news is the pattern editor will not allow you to specify a part start-up order that violates these constraints. The image below shows an example of the ordering view in the virtual system pattern editor.
Another new feature that may influence the way you build virtual system patterns is the introduction of Add-Ons. You can think of Add-Ons as special script packages that you can include in your virtual system pattern that perform system-level configuration actions. Specifically, you can include add-ons in your virtual system pattern to add an operating system user, add a virtual disk, or add a NIC during the deployment process. You include Add-Ons in your pattern by simply dragging and dropping them onto a part in your pattern, just as you do with script packages today. The difference between script packages and Add-Ons is that IBM Workload Deployer will ensure the invocation of all Add-Ons before any other scripts run during deployment. We include default Add-On implementations for adding a user, disk, and NIC.
The last new feature I want to talk about today has more to do with how you manage or govern the deployment of virtual system patterns. In WebSphere CloudBurst 2.0, we introduced the idea of Environment Profiles as a way to extend your customization reach into the deployment process. Initially, these profiles gave you the ability to directly assign IP addresses to virtual machines in your deployment, declaratively specify virtual machine naming formats, and easily split a single pattern deployment across multiple cloud groups. In IBM Workload Deployer, you will be able to use these same profiles to set resource consumption limits for pattern deployments. In particular, you will be able to set cumulative limits for virtual CPU, memory, storage, and software licenses used by deployments tied to a specific profile, thereby giving you finer-grained control over cloud resource consumption. The picture below shows the new resource limit aspects of environment profiles.
Virtual system patterns are key in the deployment model choices for IBM Workload Deployer. Not only did we carry the concept over from WebSphere CloudBurst to IBM Workload Deployer, but we made it even better. Expect this trend to continue!
I recently read a post by David Linthicum in which he proposes that a key benefit of cloud computing is the ability to transfer risk from the enterprise to the cloud provider.
At first glance, this seems an obvious benefit of using a public cloud for computing resources. Cloud providers take care of the onerous task of providing computing resources across an organization. If the resources need to be updated, require critical maintenance, or need emergency action, the cloud provider will provide those services. Enterprise IT departments are left to devote effort toward delivering technological capabilities to the business. However, does any of this imply a transfer of risk?
I'd answer that question with "It depends." Whether or not an enterprise has transferred risk by contracting with a public cloud provider depends on the provisions in the Service Level Agreement (SLA) that exists between the enterprise and provider. In some cases (maybe most) the SLA simply provides a refund for a portion of the service fee based on the impacted services. This is clearly not a case of transference of risk. The loss of current and new business sustained by the enterprise during the service outage is not indemnified by the cloud provider. In this sense, the enterprise has done nothing more than transfer the management of their risks to a third party.
True risk transference can be achieved, but it means that SLAs provide both service fee refunds and business loss indemnification. During a service outage, an enterprise's risk is not the fee they are paying for the service but instead the impact on current and future profits. There must be stipulations in the SLAs to address these losses for risk transfer to have taken place.
The differences between transferring risk and risk management may seem obvious, but it does serve to underscore the importance of SLAs in the cloud computing world. Enterprises need to fully understand these SLAs in order to accurately assess the benefits of using a cloud proider. SLAs are poised to be critical in the cloud computing world, and I'm interested to see how they will help shape the competitive landscape of the industry.
We've begun to seed this location with all sorts of helpful information on IBM Workload Deployer. Check it out and you will find links to a "getting started" section, articles, demos, redbooks, whitepapers, pointers to various blogs where authors write about private clouds or IBM Workload Deployer (yep, this blog is included), links to product documentation and education assistant, upcoming events, and more included in the wiki. We're still populating this location with content and we're looking for input on how to improve things ... so please provide your feedback and check back often to see how it evolves.
The content provided in the community is open and visible to everyone immediately. However, there is even more value if you create an id (or use your existing developerWorks id) to become a member of the community. Members can participate in the many collaborative elements that the community provides. This includes the ability to open discussions and collaborate on the forum, post blog entries in the IBM Workload Deployer community blog, or even share content that you have created which may be of interest to others.
There is even a specific section in the community focused on the Plugin Developer's Kit that Dustin mentioned in the previous post on extensibility ( see IBM Workload Deployer PDK wiki page ).
So please visit this new IBM Workload Deployer community and send us your feedback so that we can improve and grow this into a valuable resource. Ultimately, we want this to be a place where we can help each other be successful using IBM Workload Deployer. We also want to learn valuable insights from your experiences with IBM Workload Deployer so that we can continue to make improvements and optimizations in the appliance with the goal of improving your private cloud experience, making your business more agile and efficient. As always, please send us your feedback.
The concepts that govern users and user groups in WebSphere CloudBurst are fairly basic, but I get asked about them enough that I believe they warrant a short discussion. First things first, you can define users in WebSphere CloudBurst and optionally define user groups to assemble users into logical collections. For both users and user groups, you can assign roles that define the actions a particular user or group of users can take using the appliance.
All of that is straight forward, but it can get a bit tricky once we start considering the effects of user permissions when managing at the user group level. The basic premise is that when a user belongs to a group or groups, the user's effective permissions are a sum of the permissions to all of the groups to which they belong. While that is easy to say, and maybe even to understand, I feel like an example always helps.
Consider that we have a single user WCAGuy that belongs to the PatternAuthors, ContentCreators, and CloudAdmins groups. The permissions for those groups are as follows:
PatternAuthors: Users in this group have permission to create and deploy patterns
ContentCreators: Users in this group have permission to create catalog content as well as create and deploy patterns
CloudAdmins: Users in this group have permission to administer the cloud, create catalog content, and create and deploy patterns
Naturally then, it follows that the WCAGuy user can administer the cloud, create catalog content, create patterns, and deploy patterns. So then, what happens if we remove the WCAGuy user from the CloudAdmins user group? Well, as you may expect, there is an update to the user's permissions. The WCAUser user can no longer administer the cloud, but they can still create catalog content, create patterns, and deploy patterns (owing to their membership in the other two groups). Similarly, if we next removed the WCAGuy user from the ContentCreators group, then the user would retain only the permission to create and deploy patterns.
Just one more thing, let's talk about what happens when I remove a user from a group and they no longer belong to any groups. Consider that I created the WCAGuy user with the permission to create catalog content as well as create and deploy patterns. Next, I added the user to the CloudAdmins group, meaning the user now has the permission to administer the cloud. I promptly decide that the user has no business with those permissions, so I remove the user from the CloudAdmins group. What happens? The user retains the permission set of the last group to which they belonged. In this case, that means the WCAGuy user retains cloud administration rights. I have to update the user's permission set if I want to take that right away, but in this case, it will not automatically disappear upon removing them from the CloudAdmins group.
I hope this helps clear up any ambiguity you may have had concerning users, user groups, and permission sets in WebSphere CloudBurst.
When it comes to administration of WebSphere environments, I (and many others) am a big fan of scripting. In my view, any administrative action you carry out with frequency > 1 is ideally suited for a script. The downside to not using scripts (longer configuration times, inconsistent configurations, isolated expertise) is simply too steep in most cases. I also realize that simply saying that you should script is not enough. For some, the learning curve can be a bit daunting. Quite frequently, I talk about our samples gallery or provide posts with embedded scripts in the hopes that I can help flatten out this curve a bit.
While these samples can certainly help to speed up your scripting efforts for certain use cases, they are more or less helpful for solving tactical challenges when scripting. If you and your company are embarking down a strategic path that includes beefing up your administrative scripting capability, I would strongly suggest you look at a resource a few of my colleagues pointed me at recently.
The resource I am talking about is the wsadminlib.py package referenced here. This python script file is a collection of hundreds of methods that carry out common WebSphere Application Server administrative tasks. The authors carefully constructed these methods with clear method and parameter names. The result is a script resource that can become the foundation for your custom-crafted administrative scripts.
I recently downloaded the wsadminlib.py script and began constructing WebSphere CloudBurst script packages to utilize it. To say I am impressed would be an understatement. This file makes so many tasks so incredibly simple. Take for instance the creation of an SIBus. That's just a simple call like the following:
wsadminlib.createSIBus(clusterName, nodeName, serverName, SIBusName, scope, secure)
How about associating a shared library with an application or application module? Another one-line call:
wsadminlib.associateSharedLibrary (libName, appName, warName)
Or what about setting a custom property in the webcontainer? You guessed it. One-line:
wsadminlib.setWebContainerCustomProperty(nodeName, serverName, propName, propValue
This is just an extremely small sample of what the wsadminlib.py includes. As I mentioned earlier, there are hundreds of other methods that carry out various tasks including: installing applications, creating core groups, creating virtual hosts, installing BLAs, creating JMS queues, and much more. If you are looking to beef up your WebSphere Application Server scripting efforts, or if you are just starting, I strongly encourage you to look into and make use of this valuable resource!
For the next installment of this series of FAQs, let's move from product positioning and integration, square into the land of operational procedure. For this post, we will consider you are getting ready to deploy a pattern based on the WebSphere Application Server Hypervisor Edition. During the deployment process, you provide configuration information, which includes a password for a user named virtuser.
You read the documentation, and you understand that virtuser is both an operating system user and the user that WebSphere CloudBurst configures as the primary administrative user for WebSphere Application Server. Naturally, this user owns the WebSphere Application Server processes that run in the virtual machine. While it is convenient that this is all pre-configured for you, you want to know one thing: "Can I define a user besides virtuser?"
It certainly would not be the first time this question came up. The short answer to this is yes, but there are of course caveats. You can define another user and have that user own the WebSphere Application Server processes, but you cannot completely remove the virtuser user, nor should you remove virtuser as the primary administrative user. The reason for this is that WebSphere CloudBurst relies on virtuser when it carries out certain actions such as applying maintenance, applying fixes, or otherwise interacting with the WebSphere Application Server environment.
All that being said, I recently put together a script package that allows you to utilize a user other than virtuser. I hope to put the script package in our samples gallery soon, but here's a basic overview of using the script package and what it does:
Attach the script package to all parts in a pattern that contain a WebSphere Application Server process.
Deploy the pattern and provide the necessary parameter values. These include the name of the new user, a password, a common name, and a surname. The last two bits are necessary when creating a new administrative user in WebSphere Application Server.
During deployment, the script package first creates a new OS user with the specified password.
The script adds the new user to the existing OS users group.
The script creates a new WebSphere Application Server user with the same username and password and grants administrative privileges to the user.
The script shuts down the WebSphere Application Server processes.
The script changes the runAsUser value for all servers to the empty string and sets the runAsGroup value for those servers to users. This allows members of the OS users group to start the WebSphere Application Server process.
The script starts the WebSphere Application Server processes.
There are a few other activities in the script, but that should give you a basic overview. Again, note that the script does not remove the virtuser user or change that user's OS or WebSphere Application Server permissions in anyway. I would also point out that if you use WebSphere CloudBurst to apply maintenance to the WebSphere Application Server environment, it will do so as virtuser and it will restart processes as virtuser, so plan accordingly.
I hope this sheds some light on a very common question. I hope to get the sample up soon, and as always let me know if you have any questions.
I’m going to take a different approach this week in the blog. Instead of me telling you about some of the features or uses of WebSphere CloudBurst, I thought I would catch up with someone using the product everyday, WebSphere Test Architect Robbie Minshall. Robbie is responsible for a team of testers that harness a lab of over 2,000 physical machines to put our WebSphere Application Server product through some pretty rigorous testing. Toward the beginning of this year Robbie’s team started to leverage the WebSphere CloudBurst Appliance in order to create the WebSphere Application Server environments needed for their testing.
Robbie, can you tell us a little bit about what the WebSphere Application Server test efforts entail?
In WebSphere Application Server development and test we have two primary scenarios. The first is making sure that developers have rapid access to code, test cases and server topologies so that they can write code, test cases and then execute test scenarios on meaningful topologies. The second scenario is an automated daily regression where in response to a build, we provision a massive amount of WebSphere Application Server topologies and execute our automated regression tests.
Previously we have supported these scenarios through the deployment of the Tivoli Provisioning Manager for operating system provisioning, some applications for checking out environments, and then a lot of automation scripts for the silent install and configuration of WebSphere Application Server cells.
Given those scenarios and the existing solution, what are your motivations for setting up a private cloud using WebSphere CloudBurst Appliance?
We are supporting these scenarios through a pretty complicated combination of technologies. These include silent WAS install scripts, wsadmin configuration scripts, a custom hardware leasing application and the utilization of Tivoli Provisioning Manager for OS Provisioning. This solution is working very well for us though as always we are looking for areas to improve, opportunities to simplify and to reduce our dependency on investment in our custom automation scripts. Mainly, there were 3 areas where we wanted to improve our framework: Availability, Utilization and Management. This is why we started looking to the WebSphere CloudBurst Appliance.
Can you expand a bit on what you are looking for in those three areas?
The first focus area we have is availability of environments. We really wanted to lower the entry requirement for the skills and education necessary to get a development or test environment. Setting up these environments has just been too hard, too time consuming, and too error prone. Using WebSphere CloudBurst we can provide an easy push button solution for developers to get on-demand access to the topologies they need.
The second area we are looking for significant improvements on is hardware utilization. Our budgets are tight and in our native automation pools we are only using between 6-12% of the available physical resources. In order to improve this we were looking at leveraging virtualization. WebSphere CloudBurst offers the classic benefit of virtualization with the nice additions of optimized WebSphere Application Server placement and really good topology and pattern management. In our initial experiments we were able to push the hardware utilization up to 90% of physical capacity and consistently were leveraging around 70% of our physical capacity.
Finally we are looking to improve and simplify our management of physical resources and automation. We work in a lot of small agile teams and organizational priorities change from iteration to iteration. Not only does WebSphere CloudBurst allow us to maintain a catalog of topologies or patterns for releases but it also allows us to adjust physical resource allocation to teams through the use of sub clouds or cloud groups.
Basically we felt that WebSphere CloudBurst would improve the availability of application environments, enhance automation, and improve hardware utilization all with very low physical and administrative costs.
What were some of the challenges involved with getting a cloud up and running in your test department?
One of our challenges seems like it would be common to many scenarios, especially in today’s world. Our budget for new hardware to build out our cloud infrastructure was initially very limited. Most cloud infrastructure designs depict very ideal hardware scenarios including SANs, large multicore machines, and private and public networks within a dedicated lab. Quite frankly we did not have the budget to create this from scratch. It was important for us to demonstrate value and data to warrant future investment in dedicated infrastructure. After some performance comparisons we were very happily surprised to see that we could leverage our existing mixed hardware within a distributed cloud. The performance of application environments dispensed by WebSphere CloudBurst on many small existing boxes in comparison to large multicore machines with a SAN was very comparable. This allows us to leverage existing hardware, with minimal investment all the while demonstrating the value and efficiencies of cloud computing. That data in turn has allowed us to obtain new dedicated hardware to iteratively build up a larger lab specifically for use with WebSphere CloudBurst.
Specifically with WebSphere CloudBurst, are there any tips/hints you would offer users getting started with the appliance?
Sure. First, we quickly realized as we added hypervisors to our WebSphere CloudBurst setup it was critical to have someone with network knowledge on hand. This is because the hypervisors came from various sections of our lab, and we really needed people with knowledge of how the network operated in those different sections. Once we had the right people we were able to setup WebSphere CloudBurst and deploy patterns within an hour and a half.
Moving forward we continued to have challenges as we dynamically moved systems between our native hardware pool and our cloud. Occasionally the WebSphere CloudBurst administrator would move a system into the cloud but incorrectly configure the network or storage information. This lead to some misconfigured hypervisors polluting our cloud. We overcame this, quite simply and satisfactorily I may add, by creating some simple WebSphere CloudBurst CLI scripts which add the hypervisors, test them individually, by carrying out a small deployment to that hypervisor, and then move the correctly configured hypervisors into the cloud after verifying success. Misconfigured hypervisors go into a pool for problem determination. This has allowed us to maintain a clean cloud, and we are able to dynamically move our hardware in and out of the cloud to meet our business objectives.
We also use the WebSphere CloudBurst CLI to prime the cloud so to speak. Before using a given hypervisor in our cloud, we execute scripts that ensure each unique virtual image in our catalog has been deployed to each of our hypervisors at least once. When the image is first deployed to a hypervisor, a cache is created on the hypervisor side of the connection, thus meaning subsequent deployments do not require the entire image to be transferred over the wire. This gives us consistent and fast deployment times once we are using a hypervisor in our cloud.
I would assume that like many applications deployed on WebSphere Application Server, your team’s applications have several external dependencies. Some of these dependencies won’t necessarily be in the cloud, so how did you handle this?
You’re right about the external dependencies. Our applications and test cases run on the WebSphere Application Server but are dependent upon many external resources such as databases, LDAP servers, external web services etc. WebSphere CloudBurst allows us to deploy WAS topologies in a very dynamic and configurable way but the 1.0.1 version does not allow us to deploy these external resources in the same manner. This was overcome by using script packages in our patterns. These script packages allow us to associate our test applications with various patterns we have defined. The script package definition also allows us to pass in parameters to the execution of our scripts. We supply these parameter values during deploy time, and these values are used to convey the name or location of various external resources. The scripts that install our applications can access these values and ensure the application is properly integrated with the set of resources not managed by the appliance.
What is your team looking to do next with WebSphere CloudBurst and their private cloud?
The next challenge on our plate is to keep up with the demand of our expanding cloud and to develop a more dynamic relationship between our native pools and our cloud using the Tivoli Provisioning Manager. These are fun challenges to have and we look forward to sharing our progress.
I'm glad I got to spend some time with Robbie to glean some insight into their work and progress with WebSphere CloudBurst. I hope this information was useful to you. It's always nice to hear about a product from practitioners who can give you hints, tips, gotchas, and other useful information. Be sure to let me know if you have any questions about what Robbie and his team are doing with WebSphere CloudBurst.
It's here at long last! IBM PureSystems was announced today and in particular the IBM PureApplication System family member. IBM PureApplication System includes many of the capabilities that you have been hearing about and using in IBM Workload Deployer. While this solution includes and builds upon the capabilities of Workload Deployer, there's also a lot more functionality that is built into a completely integrated and optimized solution that not only manages your private cloud but runs it in the most optimized fashion. It really is a complete private cloud solution that is highly optimized to provide the best possible integration of software and hardware made simple for your cloud needs.
If you are reading this blog then I am pretty sure that you are interested in the agility that can be achieved by rapidly provisioning middleware systems and standing up virtual applications in a private cloud environment. However there are other aspects of agility that you should also consider. One such aspect is the ability to build applications that can be easily maintained, updated, and extended. This is where OSGi technology comes into the picture.
If you have been working with the IBM Workload Deployer (or watching some IBM Workload Deployer demos) you may have noticed a category of components in the virtual application builder called OSGi Components.
Maybe you already know all about OSGi applications and the value they bring to an enterprise. Or, perhaps you noticed this and decided that you would search for some more information on this odd acronym and just what an OSGi application is all about.
In a nutshell OSGi technology is a way to define dynamic modules for Java. It provides a standard way to encapsulate components (called bundles) with metadata that define versioned package dependencies, service dependencies, packages exported, services exported, etc... basically everything you need to know about this bundle so that it can be connected up with other bundles to support a particular solution. These bundles can then be grouped together into applications and dynamically wired to fulfill necessary dependencies at runtime. The OSGi framework provides all of the necessary capability to manage the dependencies and resolve any problems.
Those who leverage OSGi technology benefit from improved time-to-market and reduced development costs. The loose coupling provided by the OSGi framework reduces maintenance costs and facilitates the dynamic delivery of components in a running system. Of course there's a lot more to it than just that ... involving portability across different environments, achieving the appropriate level of isolation or sharing within an environment, and integrating with the many different technologies and patterns already available today. I don't think I know enough about OSGi to do it justice here. But fortunately for me (and you) there are several experts who can make it all clear.
One such expert is Graham Charters and there is a great opportunity to hear him introduce this topic and also participate in a dialogue about the concepts and what they mean for your business. Graham will be leading a Global WebSphere Community Lab Chat on Wednesday of this week (July 20th) entitled: How can OSGi make your enterprise more agile. Graham is the IBM technical lead in the OSGi Alliance Enterprise Expert Group and an active participant in the open source community implementing many of these standards. So register now for this free session and learn how OSGi can make your enterprise even more agile.
Over the past several months industry focus on cloud computing seems to have only intensified. Within IBM and for the purposes of this blog, WebSphere, there have been several announcements and offerings that indicate our commitment and belief in the cloud computing approach.
To further highlight WebSphere's focus and offerings in the cloud computing realm, we are embarking on a "WebSphere in the Clouds" campaign during the months of September and October. Our intent is to virtually deliver information about our cloud strategy and offerings directly from the experts to you, our WebSphere users.
The event will be kicked off by WebSphere's Director of Product Management, Kareem Yusuf, on September 23rd from 9-10 EDT. Kareem will talk about cloud computing in the enterprise, and its unique relationship to SOA thoughts and principles. In addition, he'll give an overview of what WebSphere has been doing in the cloud computing space. This will be followed by sessions from technical experts that detail WebSphere offerings in both the public and private clouds, as well as sessions that discuss enablers of application and application infrastructure elasticity.
To find out more about the "WebSphere in the Clouds" campaign, you can check out the main announcement page. To sign up for the series of virtual events visit the registration page. We hope you will join us for the series of webcasts to learn all about WebSphere's work in the clouds.
For the last post in my FAQs Revisited series, I'm going to cheat a little bit. Instead of addressing one particular question, I'm going with a grab bag of a few different questions. These are questions that I get asked quite frequently, but do not demand an entire blog post explanation. Let's get on with it.
Question: Do the new software license management capabilities provided in WebSphere CloudBurst 2.0 depend on ILMT or other supporting components?
Answer: No. The license management features are completely standalone. Of course, you can still take advantage of ILMT (through easy integration in WebSphere CloudBurst I might add) to track licenses in your cloud if you so choose.
Question: Can I deploy a pattern, make changes to my virtual system, and then recapture that as an updated pattern?
Answer: You cannot do this with WebSphere CloudBurst alone, but you can use WebSphere CloudBurst in conjunction with the Rational Automation Framework for WebSphere to do just this. Check out this article (shameless plug alert!).
Question: What if I have an urgent operating system fix to apply before IBM delivers an update to the OS in the Hypervisor Edition image?
Answer: You can either manually apply the fix to the appropriate virtual machines, or you could package up the fix as a custom WebSphere CloudBurst fix, load it into the catalog, and use the appliance to automate the application of said fix.
Question: Can I change the install location for WebSphere Application Server in the virtual image?
Answer: I've just shown you all this really cool, useful, and easy to use stuff, and you worry about install locations? Seriously though, I understand the genesis of this question usually has to do with existing scripts that assume a certain install location for WebSphere Application Server. I certainly do not advocate changing those scripts, but you cannot change the install location for WebSphere Application Server in the images. There is nothing to keep you from creating a symbolic link however.
Question: Once I deploy a pattern, what do I need to do to add more processing capacity (i.e. more application server processes)?
Answer: You have a couple of options here. You can use normal WebSphere administration techniques to add more application servers to an existing node. If that will not work (perhaps a particular node is operating at max capacity), you can use the new dynamic virtual machine operations in WebSphere CloudBurst to add an entirely new node/virtual machine. If you find yourself consistently making these types of adjustments to the runtime environment based on ebb and flow of demand, you may also want to consider the Intelligent Management Pack option for WebSphere Application Server Hypervisor Edition.
I hope this FAQs Revisited series was helpful. Stay tuned for a look at some recent work I did to integrate WebSphere CloudBurst deployments with the new WebSphere DataPower XC10 appliance.
At the core of cloud computing discussions and cloud computing in general is the idea of virtualization. The word 'virtualization' might invoke different things depending on who you talk to but for cloud discussions many people think of virtual images: entire systems being stored even down to the operating systems. The advantages seem evident. Instead of complex deployment models you simply need to take an unused piece of hardware resource and lay in the exact pattern. The assumption is that the hardware is free an compatible but everything else is negotiable. I think, however that there are different levels of assumption within the cloud concept. Laying in entire systems from the OS up may be way more work than is necessary. The advantage is that there are fewer constraints on what kinds of things you can depoy in your cloud. But one of the big disadvantages are that there are fewer constraints on what kinds of things you can depoy in your cloud. Sometimes, productivity is higher when your developers are given the topology parameters and when he knows what resources are going to be available. In fact, the concept of cloud seems to infer that there are fewer assumptions needed so you will have maximum versatility. However, flexibility is an antagonist to stability and stability is needed for prodcutivity. In effect, more assumptions necessarily equals faster developement and quicker time to release. So is cloud the antithesis of productivity? No, of course not. The beauty of clouds is that you can have as many assumptions as you want. A productive cloud model could assume specific hardware, OS and even webservers and macro-topologies. The cloud's resources could simply be avalable platforms that serve as quickly added nodes that can be dynamically provisioned within an appserver deployment. I think cloud models can be stratfied based on the number of assumptions that are built into it. Level 0 clouds could be where the only assumtion is the that the physical machines will support the virtual images. Level 6 could be that virtual servers and resources can be cloned by template to act as expansion nodes to meet growing demand. Does this cross over into other scaling models? Yes. So what? It doesn't have to be unique in every aspect it merely has to be consistent for effective use.
One of my favorite things to do is create content that you, our users, can directly use to adopt and implement our products. Luckily for me, my job allows me to spend a considerable time doing just that for our WebSphere CloudBurst Appliance. In the course of this kind of work, I use multiple different mediums to hand over what I hope is helpful content to you. This includes blogs, articles, demos, and the WebSphere CloudBurst Samples Gallery.
While I like creating content for all of these forums, if I had to pick a favorite, I'm going to go with the samples gallery every time. The reason for this is simple. Users can download and directly use the content in the samples gallery. The samples gallery plays host to script packages, CLI scripts, and other tools that are ready to use with WebSphere CloudBurst (of course, one can also extend these or simply use them as reference). Further, the samples in the gallery are mostly direct responses to suggestions or requests I get from users regarding this type of content, thus increasing its usefulness and relevance.
A good example of the kinds of assets in the gallery is the latest script package I put out there. Recently, I was talking to a user and asked, 'What do you do every single time you establish a WebSphere Application Server environment?' He outlined a few different tasks, one of those being the creation of virtual hosts in the server's configuration. The creation of virtual hosts piqued my interest because many users do that, and the configuration logic itself is fairly consistent regardless of the administrator doing the task. Therefore, I set about creating a sample script package that you can use to create virtual host configuration in WebSphere Application Server.
The script package does two things. It creates virtual host entries, and it configures host aliases for these entries. The script allows the user to supply the data for the entries and aliases they want to create via a properties file. The properties file is pretty basic and allows for the configuration of multiple host aliases for each virtual host entry. Here is an example properties file:
The script package parses the data from a properties file like the one above, and it creates the appropriate WebSphere Application Server configuration. If you are using WebSphere CloudBurst and this kind of configuration task is common for your deployments, you may want to download this free sample. I also want to point out that there are quite a few more samples that are completely free for you to download in the gallery. Check them out and let me know what you would like to see in the samples gallery!
A few nights ago, I went for a casual jog around the neighborhood. There is nothing notable about this since I do it regularly, and more to the point why should you care?? Well, it's what I did before and after the jog that may be of interest.
Sometimes when I present WebSphere CloudBurst I note a hint of skepticism in the room. It's not that the audience does not see the value in the WebSphere CloudBurst approach, since to a person they nearly always do. However, in some cases, the hardcore, impressively knowledgeable WebSphere administrators in the room are a little doubtful that they can create the same kind of heavily customized environments they rely on daily using WebSphere CloudBurst patterns. Well, I contend that you can!
I decided that rather than just make a claim, I would build a heavily customized, integrated environment in the form of a pattern. In particular, I wanted to build a pattern to accomplish the following:
Create a WebSphere Application Server cell augmented with WebSphere Virtual Enterprise management capabilities - including an On Demand Router
Create a dynamic cluster
Install an application into the dynamic cluster
Configure HTTP session replication behavior backed by the WebSphere DataPower XC10 Appliance
Create a DB2 database instance complete with a database and table
Create a WebSphere Application Server data source to the DB2 instance
I would consider this a fairly complex target environment with multiple integration points (XC10 and DB2). In WebSphere CloudBurst, I started by creating a custom virtual image and installed the necessary WebSphere eXtreme Scale binaries. This provided me the basis to achieve integration with the WebSphere DataPower XC10 Appliance. After creating the image, I built the pattern pictured below.
With a combination of parts, scripts, and advanced options provided by WebSphere CloudBurst (to setup my dynamic cluster), the pattern above encapsulates each one of my goals from above. And, before you throw your hands up and tell me how much scripting that is, the longest script is less than 50 lines! Better yet, I only have to write these scripts once, and I only have to provide invocation instructions once. Once completed, the invocation of my scripts is an automated process. This means I cannot muck up the configuration work.
So, what does all of this have to do with my jog? On the way out of the house that evening, I hit the deploy button for the above pattern. When I walked back in the house, I logged into WebSphere CloudBurst, pulled up my virtual systems and saw it was up and running. I logged into the WebSphere Application Server administration console and verified that the resulting environment met every last one of my deployment goals. How is that for low-touch? I simply clicked the deploy button, went for a run, came back and I had a pretty interesting application environment up and going.
During the week of IMPACT this year, we announced the launch of the WebSphere CloudBurst Samples Gallery. You can go to this gallery to find and download sample script packages, CLI scripts, and other tools that we hope help you in your endeavors with the appliance. The samples are free to use and offered in an "as-is" fashion.
While I certainly will not write about each and every sample we post out there, I did want to bring your awareness to a new one I just put up today. The new sample is neither a CLI script nor a script package, though you will find it in the script packages section of the gallery. Instead, the new sample is a tool that you can run to produce WebSphere CloudBurst script packages.
Specifically, the tool runs against a target WebSphere cell to produce a WebSphere CloudBurst script package that encapsulates that cell's configuration. The tool works by running the backupConfig command against the target cell. It packages the ZIP file that results from running the command into a special WebSphere CloudBurst script package that you can include in patterns which match the source cell in node quantity and type.
The script package produced by the tool packages logic to run the restoreConfig command using the backed up configuration from the source cell. This will apply the source configuration to a new WebSphere Application Server cell created as the result of deploying a pattern. In addition, the script package contains logic to handle the possibility of changing cell, node, and host names in the target environment.
The tool’s purpose is to help you accelerate the process of importing your existing WebSphere Application Server environments into the appliance as patterns (which is a problem I believe many of you would like to solve). It certainly does not handle everything you need to do to import environments. In fact, it has the same limitations as the backupConfig/restoreConfig utilities in WebSphere Application Server. However, I do believe that it makes it a little easier to start moving your existing environments into the appliance as new WebSphere CloudBurst patterns.
Check out this video to see a quick overview of the tool, and then go download it for free from the samples gallery. The ZIP file that you download has a readme file that gives specific detail about how to use this sample tool. As always, please let me know if you have any questions or feedback.
In the course of my job, I am lucky to be able to work with both enterprise users and business partners who are adopting and using the WebSphere CloudBurst Appliance. When it comes to the business partner camp, one of my absolute favorites is the Haddon Hill Group. The Haddon Hill Group is an IBM Premier Business Partner, and they have been an early adopter and vocal advocate of the WebSphere CloudBurst Appliance. They have extensive knowledge of the use of the appliance in enterprise accounts, and quite frankly, they are doing some really cool things with WebSphere CloudBurst.
Given the above, I was glad to see summarized results from their various implementations made available recently on the IBM site. The summary is fairly concise, so I encourage you to take a look at the results Haddon Hill Group is getting with WebSphere CloudBurst.
I am not going to rehash the contents of the results here, but there are a couple of things I want to call out. First off, Haddon Hill Group says that WebSphere CloudBurst can provide companies with a '100 times faster time to market' delivery experience. In a practical sense, this means reducing the amount of time to deliver WebSphere environments from 40-60 days on average to just hours. That is an eye-opening data point!
The other thing I want call out here is a quote from Phil Schaadt, President and CTO, Haddon Hill Group. I have had the pleasure of working with Phil and team, and I have heard him echo these same sentiments many times:
"The important thing about the IBM WebSphere CloudBurst Appliance is that it will dispense a WebSphere Application Server image onto your WebSphere Application Server environment or private cloud along with other products within the WebSphere stack, and that application server will be ready in a few minutes. You can do it in a clustered environment, and you can even roll out IBM WebSphere Process Server and get it right in a fully clustered environment with a database connection, in about 90 minutes. You can also easily manage all the configurations of IBM WebSphere Process Server that you need. All the steps that took up so much time and effort on the part of IT staff have been removed. The savings for companies with large WebSphere implementations can be in the millions."
It is always great to see clients putting our technology to use to produce tangible business value. Again, I encourage you to take a look at these reports. As always, I am eager to hear what you think, so leave me a comment or reach out to me on Twitter @damrhein.
One of the most powerful features of WebSphere CloudBurst is the ability to take one of the WebSphere Application Server Hypervisor Edition virtual images that are shipped with the appliance and extend it to a produce a custom virtual image. This allows users to begin creating customized environments from the bottom up, starting with the operating system. To put it in better context, let's take a look at a couple of scenarios where this feature comes in quite handy.
First off, a very common need for our customers is the ability to continually monitor their application environments. For instance, you may install Tivoli monitoring agents on all of your machines hosting WebSphere Application Server processes and configure those agents to report back to a management server. This is a great case for image extension in WebSphere CloudBurst.
In this scenario, you would start by extending an existing WebSphere Application Server Hypervisor Edition image. WebSphere CloudBurst creates a running virtual machine based off of the selected image, and you log into that virtual machine and install the Tivoli monitoring agents. Once the installation is done, you capture the virtual image back into the WebSphere CloudBurst catalog and use the new image to build a custom pattern. The last step is to include a script package on this custom pattern that, upon deployment, will configure the installed monitoring agents to report back to your desired management server.
Another use case is likely to be of interest to you if you are using WebSphere Virtual Enterprise (or something similar), and you could benefit from the same ease of provisioning for those environments that WebSphere CloudBurst brings to WebSphere Application Server environments. You can use the same customization combination above (image extension and custom scripts) to enable WebSphere CloudBurst to essentially dispense WebSphere Virtual Enterprise cells.
Again, this scenario starts off by extending a WebSphere Application Server Hypervisor Edition virtual image. Once the virtual machine for the extension is created by WebSphere CloudBurst, you log in and install the WebSphere Virtual Enterprise product. After the installation is done, you capture the image and store it in the catalog. Next, you build a custom pattern based off of this image and include script packages that, upon deployment, augment the various parts in the pattern from WebSphere Application Server profiles to WebSphere Virtual Enterprise profiles. (You may wonder why you wouldn't just create the WebSphere Virtual Enterprise profiles during the image extension process. This is because during image extension, you cannot make changes to the virtual disk that contains the WebSphere Application Server profiles. Any changes made to the profiles will be wiped out during the capture process.)
There are countless more scenarios for creating custom virtual images in WebSphere CloudBurst. To name a few, you may want to install JDBC drivers that are common to almost all of your application environments, install required anti-virus software, or just make operating system configuration changes. All of these things can be accomplished through the image extension and capture process. Look for an article coming out soon that will discuss and explain, in much greater detail than I provided here, the process of installing and configuring Tivoli monitoring agents in environments dispensed by WebSphere CloudBurst. In the meantime, if you have any questions or comments, drop us a line here or check out our forum.
Dustin and i manned the IBM booth at InterOp in Las Vegas last week. The conference was very different from the industry conferences I remember, but then again I haven't been to one in a long time. I don't recall seeing boxing matches, light shows or bikini models but I think they are a welcome addition.
Ostensibly this conference was focused on cloud computing and was even called the "Cloud Summit". However, in the vendor area, there were few real cloud computing specific peds. Most of the vendor displays were about hardware, system monitoring and security.
Ric Telford of IBM gave a keynote address and sspoke of IBM's cloud offerings. After the keynote, there was a flurry of visitors asking about IBM, Cloud Computing and IBM's cloud offerings. Most of the visitors were looking for education and we were happy to have the opportunity to talk about the company and cloud computing from IBM's perspective.
We had the CloudBurst Appliance with us and it drew some interest. The purple case definitely stood out and drew inquiries. Some excitement is being generated but I think now the industry and the market has to catch up to us.
Customization capabilities have been very important to the design of IBM Workload Deployer going back to the beginning with WebSphere CloudBurst. Having the ability to quickly spin up environments in a cloud really does little good if those environments are not customized according to your needs. If you look at the virtual system pattern capability, it is why we always had the notion of custom images, custom patterns, and custom scripts. We give you a strong foundation, and you tweak it here and there to create what you want.
Customization is not a concept unique to virtual system patterns. The virtual application model in IBM Workload Deployer supports many different mechanisms for you to tailor your cloud-based environments. You can start with the virtual application pattern types that we ship and use any components in those patterns to build a custom environment. The patterns you build can include your own configuration (within the set of configurable parameters) and include policies that you need for your environment. In looking at just the IBM Workload Deployer Pattern for Web Applications and the IBM Workload Deployer Pattern for Databases, there are quite a number of scenarios you can support with your cloud. However, what happens when you want to go a little further and color outside the lines of what we provide?
At some point you may have heard or read that the entire virtual application pattern model resides on a pluggable architecture. In effect, this means that everything about a virtual application pattern type, from the elements that show up when building a pattern to the management interface you interact with after deployment, is customizable. The fundamental unit of customization for a virtual application pattern type is a plugin. Plugins provide the know-how in terms of installing, configuring, integrating, and managing the application types supported by a given pattern. Plugins also provide metadata that control what users see as they build and manage these patterns. In short, plugins are the source of truth for virtual application patterns!
If you looked in IBM Workload Deployer, you would find the collection of plugins that support the virtual application pattern types shipped with the offering. While that is interesting, you should also know that you can supply your own plugins. That's right. You can develop a plugin, and load it directly into the appliance. This allows you to do two very important things. First, you can extend the virtual application pattern types that come with IBM Workload Deployer with any kind of functionality you deem important. This may be additional monitoring, integration with external systems, or any number of other extensions. Second, you can create new virtual application pattern types that support your desired workloads. You can support the workloads with the software of your choosing so long as you can supply the necessary know-how in your plugins. In either case, you contribute the plugin, and your customized components become first class members of the IBM Workload Deployer landscape.
Okay, so I admit that this is not necessarily news. We have supported user-contributed plugins since the release of IBM Workload Deployer. However, there is something new that significantly lowers the barrier to entry in the custom plugin game. Early last week, IBM announced the IBM Workload Plugin Development Kit. This kit provides a set of tools and samples designed to make the construction and packaging of custom plugins a simple process. In my opinion, this reiterates our commitment to an extensible, application-centric cloud approach, and it represents a huge step forward in the industry as a whole. Be sure to check this out, and don't be shy with the comments and feedback!
I'm out at the RSA conference in San Francisco this week, and I'm expecting a lot of good conversations about WebSphere CloudBurst and security. This topic always comes up when I'm out and talking to customers, and I approach it from a few different angles.
First of all, WebSphere CloudBurst enables the creation of on-premise clouds (clouds in your data center). This means that you retain control over the resources that make up and support your cloud, and you have the ability to very tightly secure said resources. Notice that I say "you have the ability". I'm careful to point out that on-premise clouds do not inherently make your environment secure. If you don't already have a robust security strategy in place within your enterprise, then simply moving to a cloud model will not solve much. That being said, if you do have a comprehensive security strategy in place, one built around customized processes and access rights, then on-premise clouds are likely to make much more sense for you.
Moving beyond the opportunity for customized security controls provided by on-premise clouds, WebSphere CloudBurst delivers additional, unique security features. It starts on the outside with the tamper-resistant physical casing. If a malicious user attempts to remove the casing to get to the inner contents, the appliance is put into a dormant state, and it must be sent to IBM to be reset. "So what!" you say. If the user removes the casing and gets to the contents, couldn't they simply read the contents off the flash memory or hard disks directly, or insert them into another WebSphere CloudBurst Appliance and read them from there? Nope. All of the contents stored on the appliance's flash memory and hard disks are encrypted with a private key that cannot be changed and is unique to each and every appliance.
If you are at all familiar with WebSphere CloudBurst, you know that the appliance dispenses and monitors virtual systems running on a collection of hypervisors. Obviously then, the appliance must remotely communicate with the hypervisors. In order to secure this communication, all information between WebSphere CloudBurst and the hypervisors (and vice versa) is encrypted. This encryption is achieved by using an SSL certificate that is exchanged when a hypervisor is defined in WebSphere CloudBurst. This certificate must be accepted by a user, thus preventing rogue hypervisors from being defined in WebSphere CloudBurst.
Finally, WebSphere CloudBurst provides for the definition of users and user groups with varying permissions and resource access rights in the appliance. You don't have to turn over the keys to your cloud kingdom when you add a user to the appliance. You have the capability to define varying permissions (from simply deploying patterns, to creating them, all the way up to administering the cloud and appliance), and you have the ability to control access to resources (patterns, virtual images, script packages, cloud groups, etc.) at a fine-grained level. These two capabilities combine to allow you to control not only what actions a user can take, but also on which resources they can take those actions.
WebSphere CloudBurst was designed with focus on delivering a secure cloud experience, and I think it hit the mark. I'm sure I didn't address all your WebSphere CloudBurst and security related questions. If you have something specific in mind, leave a comment on the blog or reach out to me on Twitter. I'll do my best to address your question.
Can you have cloud computing without virtualization? I don't think so. Some have stated to me that they believe a cloud could b contrived without virtualization but I don't see it. Cloud computing is all about being able to expand or contract an enterprise on demand and as a service. Without deployable virtual images there is no mechanism for doing this efficiently.
I suppose that you could argue that clouds do not necessarily include the OS or the hardware and that you could scale by use of templates and configuration files to clone systems. That is cloning. Cloning, spawning, parallel processing and other mechanisms for creating capacity and processing power dont have the scope that a full cloud implementation has. Cloud computing is an administration paradigm that may share or even employ some or all of these other techniques but can include more.
I guess the biggest thing that sets cloud computing apart from cloining or spawning is that cloud computing is a paradigm for a flexible distributed computing platform. Cloning and spawing are techniques as is Virtualization.Clouds are entire managed infrastructures where virtulized systems are simply tools and cloning is a function of products.