Usually when I am discussing WebSphere CloudBurst with clients, the subject of tracking usage comes up. While 'tracking usage' is pretty broad and could apply to any number of things, we often come back to two major concepts. First, users want to be able to track compute resource usage in the WebSphere CloudBurst cloud, as this helps in cloud capacity planning. Second, users want to be able to track usage by individual WebSphere CloudBurst users in order to facilitate chargeback. In both cases, WebSphere CloudBurst provides reports that help you.
When it comes to tracking compute resource usage in your WebSphere CloudBurst cloud, the appliance provides a set of pre-defined reports on the Cloud --> Machine Activity page.
As you can see from the snapshot above, WebSphere CloudBurst provides usage reports for both memory and CPU attributed to either individual hypervisors or virtual machines. In addition, the appliance tracks storage usage by device and IP usage in your cloud. For each report type, you can specify a desired date range and let WebSphere CloudBurst produce a graph showing usage over that time. The below picture shows the report for memory usage by hypervisor over a one month period.
Tracking compute resource usage is certainly important, but if your interests are mostly about using WebSphere CloudBurst to facilitate chargeback, you likely want to know about our user reports. You can find these reports on the Cloud --> User Activity page of the appliance. On this page, you will find a table that lists each user and their usage of virtual machines, CPUs, memory, and storage over a period of time that you specify. Further, you can download a comma separated value file that contains this data for further parsing or processing on your part. The image below shows an example of the user activity table.
In addition to the user usage data provided above, many WebSphere CloudBurst users find that they want to track the amount of time users had running virtual systems deployed through WebSphere CloudBurst. While the appliance does not provide a direct report with this information, you can use this free sample to calculate virtual system duration times. This free tool uses data available in the WebSphere CloudBurst audit log (data you can process to produce any custom report you need), and it calculates virtual system duration time as well as virtual system time attributed to each user. You use the WebSphere CloudBurst CLI to invoke this tool, providing it with your start and end dates for the calculation (you can find further invocation instructions inside the ZIP file containing the tool). The image below shows example output for both the virtual system duration and user virtual system time reports.
'Tracking usage' means many different things to different people and use cases. I hope the above information regarding usage tracking in WebSphere CloudBurst gives you a good idea of what you get out of the box, as well as what you can do by using the audit log (in a similar fashion to the free tool mentioned above). If you have any questions, requests, or feedback, please let me know.
Application-centric cloud computing is the main thrust behind the new capabilities of IBM Workload Deployer v3.0. But what does that really mean? After all, application-centricity is really just a concept. Granted, it is an important concept, but it is fairly meaningless until it is put into action or implemented. IBM Workload Deployer does just that with its new Virtual Application Patterns (VAPs).
VAPs are the embodiment of the workload pattern approach I briefly discussed in an overview post a few weeks back. The idea with a VAP is to give the user an interface through which they can provide their application, specify dependencies, declare functional and non-functional requirements and then deploy. Of course application middleware is a part of the overall solution, but IBM Workload Deployer has the smarts to build, configure, and integrate the necessary infrastructure in order to support the user's application. This is completely hidden from the user, so they are liberated to focus on the application and its requirements.
If we scratch a bit further beneath the surface of a VAP, we see that these patterns contain three primary pieces. These primary pieces are components, links, and policies, and they are fundamental to understanding how virtual application patterns work. Let's start with the building blocks of VAPs, components. Put simply, components represent different resources and functionality profiles that make up your application environment. As an example, the IBM Workload Deployer Pattern for Web Applications is a VAP that contains components for an EAR file, WAR file, message queue, and any number of other components that are typical requirements for a web application. The components will certainly vary based on the workload type (i.e. the components included in a web application VAP would be different than those included in a batch application VAP), but they are the foundation of any VAP.
From the ground up, the next logical element we come to in the VAP is a link. A link is a way to declare a dependency or integration point between two components. As an example, consider a VAP with a WAR file component and a database component. You might draw a link between the WAR component and the database component to indicate that your web application uses or otherwise depends on the database. IBM Workload Deployer interprets this link, and takes it as a directive to configure the integration between the two components as a part of deployment. In this case, that may mean configuring a data source in the application's container. This is just a simple example, and an application may have any number of links between components.
Finally, we come to the policy element within the VAP. A policy is a way for a user to specify functional and non-functional requirements for their application environment. Users attach policies to the VAP, or to components in their VAP, and IBM Workload Deployer interprets and enforces those policies. In the context of a web application, one example of a policy could be a scaling policy. The scaling policy might indicate scaling requirements for the application that included minimum application instances, maximum application instances, and conditions that triggered scaling activities. IBM Workload Deployer would use the information in a scaling policy within a VAP to appropriately manage the deployed, running environment. Other examples of a scaling policy may include a JVM policy that provides configuration directives for the java virtual machines in your application environment or a logging policy that defines logging configuration options. In any case, the policy element allows VAP builders to influence the configuration and management of the application environment.
In the example VAP below you can see the use of components (Enterprise Application, Database, User Registry, Messaging Service), links (blue lines between components), and policies (Scaling Policy, JVM Policy):
In total, when I look at a VAP a particular word sticks out to me: declarative. VAPs really enable declarative, application-centric cloud computing. What do I mean? By declarative, I mean you are telling IBM Workload Deployer what you want, but not necessarily how you want it done. It is the job of IBM Workload Deployer to take care of the how. This shift in approach to application environments enables the potential for significant savings, and more importantly to me, lays the foundation for a more agile, flexible approach to deploying and managing application environments.
There will be more in the weeks and months to come on IBM Workload Deployer, so stay tuned. I also want to put a plug in for a new blog from Jason McGee. For those that do not know Jason, he is an IBM Distinguished Engineer, and the lead architect behind IBM Workload Deployer. Be sure to check out his blog for insights on this new offering, as well as for all things cloud.
Over the past several months industry focus on cloud computing seems to have only intensified. Within IBM and for the purposes of this blog, WebSphere, there have been several announcements and offerings that indicate our commitment and belief in the cloud computing approach.
To further highlight WebSphere's focus and offerings in the cloud computing realm, we are embarking on a "WebSphere in the Clouds" campaign during the months of September and October. Our intent is to virtually deliver information about our cloud strategy and offerings directly from the experts to you, our WebSphere users.
The event will be kicked off by WebSphere's Director of Product Management, Kareem Yusuf, on September 23rd from 9-10 EDT. Kareem will talk about cloud computing in the enterprise, and its unique relationship to SOA thoughts and principles. In addition, he'll give an overview of what WebSphere has been doing in the cloud computing space. This will be followed by sessions from technical experts that detail WebSphere offerings in both the public and private clouds, as well as sessions that discuss enablers of application and application infrastructure elasticity.
To find out more about the "WebSphere in the Clouds" campaign, you can check out the main announcement page. To sign up for the series of virtual events visit the registration page. We hope you will join us for the series of webcasts to learn all about WebSphere's work in the clouds.
Virtual image parts play a huge role in WebSphere CloudBurst. When crafting your own customized patterns, you include anywhere from 1 to n parts from as many different virtual images as is necessary. These parts represent the different node types or personalities within a given Hypervisor Edition image, and form the basis of your pattern. When you deploy a pattern, such as the one pictured below, WebSphere CloudBurst creates a distinct virtual machine for each part.
This means that after deploying the above WebSphere Application Server pattern, you will have four virtual machines comprising your virtual system. This gives you a clean separation of concern by providing a unique container for each of your application environment nodes. This can attribute to performance optimization, increased availability, and much more. However, this approach is not suitable to all use cases. In some scenarios, especially when trying to control costs and increase consolidation, you may want to deploy a multi-node WebSphere Application Server environment within a single virtual machine. Based on what I showed you above, you might think our approach in WebSphere CloudBurst makes this impossible, but you would be overlooking an important component of patterns.
That component is of course the second building block of patterns... script packages. As you probably know, script packages allow you to supply just about any customization you want. In the case that you want a single virtual machine to host a number of WebSphere Application Server nodes, maybe even an entire cell, all you need to do is supply a script package that constructs the necessary nodes during deployment. In fact, you don't even have to write the script package. You can use the free sample in our samples gallery. As seen in the pattern below, you include this script package on a sole deployment manager part in a pattern.
The script script package provides parameters that define the node name, number of custom nodes, and number of web server nodes you want in your cell. During the deployment process, the script takes this information and constructs the cell you define. This includes creating the custom and web servers nodes and federating the custom nodes, thus completing the creation of your WebSphere Application Server cell. In this case, the script package provides deployment flexibility that is sometimes a necessity, and it is just another example of the many degrees of flexibility enabled by the script package design.
I should point out that a part in a pattern does not always map to a single node. For instance, in the case of WebSphere Process Server, there is a part that represents a complete, multi-node golden topology encapsulated within a single virtual machine. However, if you find yourself using images that do not contain these multi-node parts, rest easy knowing script packages provide you the flexibility you need.
I point this out about script packages because recently I put one together that is not a WebSphere Application Server administration task, but does provide configuration logic common to many WAS deployments. Specifically, I put together a script package that configures an IBM HTTP Server to be a reverse proxy server. This of course, allows clients to send requests to the IBM HTTP Server and have those requests pass through to a specified back-end destination (i.e. a service hosted on WebSphere Application Server) based on URI paths.
The script package is not all that different from many of the ones I put together. It contains a shell script (which provides configuration and orchestration logic) as well as a cbscript.json file that defines the script package's characteristics when I upload it into WebSphere CloudBurst. The notable difference in this script package is that I include a Perl script that modifies the IBM HTTP Server configuration file. This just reiterates the point that you are not limited to only wsadmin and shell scripts within your script packages.
The workings of the script package are quite straightforward. It starts with a call to the shell script that modifies the IBM HTTP Server's configuration file to ensure the loading of a couple proxy modules:
sed -i s/"#LoadModule proxy_module modules\/mod_proxy.so"/"LoadModule proxy_module modules\/mod_proxy.so"/g $HTTP_CONF
sed -i s/"#LoadModule proxy_http_module modules\/mod_proxy_http.so"/"LoadModule proxy_http_module modules\/mod_proxy_http.so"/g $HTTP_CONF
As you can see, the createProxy.pl script accepts a single argument. This argument represents the reverse proxy configuration information provided by the user during deployment. The Perl script parses the single argument and creates the appropriate proxy directives in the IBM HTTP Server's configuration file:
After the invocation of the script above, control returns to the shell script. The shell script restarts the IBM HTTP Server so that the configuration changes take place. The result is an up and running IBM HTTP Server acting as a reverse proxy based on information supplied during deployment. The listings here do not show the full script package, but I hope to have it up on our WebSphere CloudBurst Samples Gallery soon.
If the script above provides some configuration logic you can use, that is good. However, my main point for bringing it up here is to point out that WebSphere CloudBurst script packages can be more than shell and wsadmin scripts that perform WebSphere Application Server configuration tasks. You can use them to do any sort of scripted activity that is essential to your application middleware deployment process. Happy scripting!
One of the most exciting announcements at IBM IMPACT last week was that of the new WebSphere Process Server Hypervisor Edition. This new virtual image allows you to provision complete WebSphere Process Server environments into your on-premise cloud using the WebSphere CloudBurst Appliance. Just like with the other environments you can provision using WebSphere CloudBurst (namely WebSphere Application Server, DB2, and Portal Server), you can stand up these WebSphere Process Server environments in a matter of minutes.
The WebSphere Process Server does not come pre-loaded on the appliance, but it does come with a cool utility that helps you get it on the appliance. The WebSphere Process Server Hypervisor Edition loader provides a wizard-like tool that loads the image into the catalog of an appliance you specify. The tool is simple to use and is included as part of the image package that you download from Passport Advantage.
Not only does the loader above populate the WebSphere Process Server Hypervisor Edition into the appliance's catalog, but it also creates a set of patterns for the WebSphere CloudBurst Appliance. These patterns encapsulate golden topology environments for WebSphere Process Server Hypervisor Edition. At the time of my post, the patterns created by the loader include the following:
Standalone server: This pattern represents a single server instance of WebSphere Process Server. Deployment of the pattern results in a single virtual machine that contains both the server instance and a DB2 instance.
Simulated environment: This pattern contains a single part called a 'Full function control node'. Deployment of the pattern results in the creation of a deployment manager, proxy server, DB2 environment, and three WebSphere Process Server clusters (application target cluster, support cluster, and messaging cluster), all in a single virtual machine.
Scalable environment: This pattern contains a deployment manager, 'Basic function nodes' part, DB2 part, and a proxy server. Deploying the pattern results in the same components as the pattern above, but in this case each component resides in its own virtual machine.
The announcement of the WebSphere Process Server Hypervisor Edition only serves to increase the applicability of WebSphere CloudBurst for constructing on-premise WebSphere clouds. If you have any questions, or want to learn more about this new virtual image, please let me know.
I spent most of my time growing up doing two things, going to school and playing sports. I made many fond memories -- mostly from the latter :) -- and learned more than a few lessons over that time. Of all of those lessons, there was one in particular that stuck out in both the classroom and on the baseball diamond: Sometimes you have to get back to the basics.
In that vein, I think it is time to revisit the basics of WebSphere CloudBurst. In revisiting the basics, I am not talking about the technical basics of the appliance. Rather, I am talking about revisiting exactly why WebSphere CloudBurst exists in the first place. In other words, let's take a look at the problem domains WebSphere CloudBurst addresses, and let's discuss a little bit about how the appliance does so.
If you follow this blog often, you know that from time to time I like to post frequently asked questions. Well, it's been a while since I have done that, and since then I have added some new questions to my list -- along with some regulars. Take a look below, and if I don't answer your question feel free to leave a comment!
Can IBM Workload Deployer deploy software that is not IBM software? Yes. You can use one of the included images as a springboard and customize them with your own software via extend and capture. Additionally, you can use the IBM Image Construction and Composition Tool (I'm getting ahead of myself here) to create your own custom images from the ground up and use those within IBM Workload Deployer.
Can I use VMotion for the systems I deploy with IBM Workload Deployer? Yes. IBM Workload Deployer has tolerated the use of VMotion since the WebSphere CloudBurst days (see the Additional Considerations section on this page for more information). IBM Workload Deployer v3 introduced the notion of virtual machine mobility initiated directly from the appliance. This capability takes advantage of VMotion in the case of VMware-based cloud environments.
Can IBM Workload Deployer deploy just a base operating system? Yes. IBM Workload Deployer v3 introduced a base operating system image that contains 64-bit Red Hat Enterprise Linux. Internally, IBM Workload Deployer uses this as the foundation on top of which virtual application patterns are deployed. You can use it to deploy virtual machines containing just the base OS, or you can customize it to deploy software of your choosing. (As an aside, IBM Workload Deployer v3.1 will include a base operating system image for AIX)
Can I automate the process of calling/using IBM Workload Deployer? Yes. IBM Workload Deployer is built to fit a specific need -- creating and managing a cloud of middleware and middleware-based workloads. In that light, it would be a shortcoming if IBM Workload Deployer did not to fit well into more holistic or enterprise-wide cloud management systems. The REST API and CLI allow you to automate the use of IBM Workload Deployer, thereby allowing it to be mashed up into other processes.
Can I group two appliances together for high availability? Yes. IBM Workload Deployer v3.1 introduces the ability to configure appliances in a master/slave setup. You can connect two appliances, allow them to share a floating IP address, and be confident that data is continuously replicated between the two. If one appliance fails, the other appliance picks up the floating IP ensuring continuous service.
Are images created using the Image Construction and Composition Tool supported for use within IBM Workload Deployer? Yes. Part of the new IBM Workload Deployer 3.1 announcement was a statement of support for using images created by the Image Construction and Composition Tool as a component of your virtual system patterns. This is a very important enhancement as it allows you to extend the set of content deployed by IBM Workload Deployer while being sure that you are operating within the boundaries of intended use.
Can I use IBM Workload Deployer to provision to public clouds? No... and yes. If you install an IBM Workload Deployer appliance in your datacenter, you cannot use it to deploy to a public cloud environment. However, you may have recently heard about the IBM SmartCloud Application Services portfolio. IBM has announced that the pattern-based provisioning that one gets with IBM Workload Deployer will also be available as part of this portfolio. This means that you will be able to build and deploy patterns using a service hosted on the IBM SmartCloud. Further, your deployed systems will run on the IBM SmartCloud. Check out this demo for more information.
** IBM Workload Deployer 3.1 firmware is available on 11/18.
A couple of weeks ago, I dropped by the Intel Developer Forum to present a session and listen in on a few others. As always in these types of shows, I learned quite a bit. Most strikingly though, I was reminded of something that is probably quite obvious to many of you: Consumer interest in cloud computing will not be letting up any time soon.
Based on this, and some of the other things I heard at the show, I decided to catch up with fellow IBMer Marc Haberkorn. Marc is an IBM Product Manager and is responsible for IBM Workload Deployer amongst other things. I asked him about IBM Workload Deployer, the competition, and cloud in general. Check out what Marc had to say below:
Me:IBM Workload Deployer is one among many of a growing wave of cloud management solutions. How do you differentiate the focus and business value of it versus the myriad of other solutions out there?
Marc: To sum it up, we offer a combination of depth and breadth. IWD delivers both workload aware management and general purpose management. Workload aware management differentiates IWD from its competition, as it can deliver more value for the set of products for which it has context. There is a set of actions that workload aware management tools can do that is normally left to the user by general purpose management tools. This list includes configuring a middleware server to know its hostname/IP address, configuring multiple middleware servers to know of one another, arranging clusters, applying maintenance, and handling elasticity. By handling more of these activities in the automated flow, there are fewer chances for manual errors and inconsistencies to enter a managed environment.
That said, without infinite resource or time, it’s impossible to deliver this context-aware management for everything under the sun. As such, in order to allow IWD to deliver differentiated value AND allow it to handle a customer's entire environment, we offer a mix of workload-aware management and general purpose management.
Me:VMware is a good example of a company active in the cloud space, and they seem to keep a consistent pace of new product delivery. What do you think of their product development focus?
Marc: I think VMware has built a very compelling set of capability in the virtualization space. I think the main difference between VMware's suite and IBM Workload Deployer is the perspective from which the environments are managed. VMware puts the administrator in the position of thinking about infrastructure from the ground up. The administrator is thinking about virtual images, hypervisors, and scripts. In IBM Workload Deployer, we think about things from the perspective of the app, because that's ultimately what the business cares about. By providing a declarative model through which an application can be instantiated and managed, we feel we deliver a deeper value proposition to clients, through workload-aware management.
Me:The 'one tool to do it all' approach is a popular, if not hard to achieve goal. What is your advice to users when it comes to choosing between breadth and depth for cloud management solutions?
Marc: The advantages of a "one tool to do it all" are many: less integration, more uniformity, less complexity. As such, customers will always prefer a single tool when possible. This is why IBM Workload Deployer has focused on not only providing differentiated, deeper value for common use cases but also providing a way to handle the "everything else." As such, my advice to users is not to choose between breadth and depth - use IBM Workload Deployer which offers both.
Me:To close, I'm curious to know where you think we are heading in the cloud market. What do you think users will be most readily adopting over the next one to two years? Where does the cloud industry need the most innovation?
Marc: I think most users are currently looking at the broad picture of cloud computing, and have been adopting primarily in the private cloud realm. There are several reasons for this. One reason is that many customers have a large set of hardware resources which amount to sunk cost that needs to be leveraged. Another reason is around data security concerns in off-premises clouds, and still another reason is around the human factor of comfort, which has taken time to develop around off-premise cloud models. However, businesses have become increasingly comfortable with various sources of outsourcing in recent years, especially in mission critical areas involving very sensitive data. Just look at IBM's Strategic Outsourcing business, which handles entire IT operations for many large businesses. I think that trend will (and really, has already begun to) continue in the area of cloud computing, and will lead to more public and ultimately hybrid cloud computing adoption. In order to get to hybrid cloud computing, I see much of the focus and innovation being associated with data security, workload portability (across private and public, in a seamless fashion), and license transferability between private and public. When this space reaches fruition, clients will be able to enjoy true elastic economics in a computing model that allows a mixture of owning and renting compute resources and software licenses.
In my last post, I concentrated on the new enhancements to WebSphere CloudBurst 22.214.171.124. One of the major new additions was the introduction of Environment Profiles, and I promised a developerWorks article would be forthcoming. The article is now live along with a demo that showcases the capability of environment profiles.
As I mentioned in my last post, environment profiles center around giving you more customization capability during the pattern deployment process. In WebSphere CloudBurst, the pattern deployment process consists of the five main steps depicted below.
Traditionally, WebSphere CloudBurst controlled the entire deployment process, thus closing it off to the deployer. Environment profiles extend the customization reach of users to be able to effect steps 1-3 in the above diagram. Specifically, environment profiles give you the following control:
Control over the assignment of IP addresses and hostnames to pattern parts: Instead of having WebSphere CloudBurst automatically assign IP addresses, and thus hostnames, to virtual machines during deployment, you can explicitly set both values during the deployment process.
Ability to deploy single patterns to multiple cloud groups: Previously, when deploying a pattern you selected a single cloud group and WebSphere CloudBurst deployed all the parts in the pattern to machines within that cloud group. While this may be okay for many cases, other cases may require you to deploy some parts of the pattern to one group of machines while other parts map to a separate set of machines. Before environment profiles, you could accomplish this with multiple patterns. With environment profiles, you can accomplish it with a single pattern.
Ability to supply virtual machine naming standards: As part of deploying a pattern, WebSphere CloudBurst creates one to many virtual machines with distinct names. Environment profiles allow you to supply a naming standard that WebSphere CloudBurst will use when creating the machines as opposed to default naming schemes previously used.
It is important to note that the use of environment profiles is completely optional, and you can continue to use the traditional deployment process, thereby leaving WebSphere CloudBurst in control. That said, the introduction of environment profiles is a direct response to consistent user feedback we received regarding the need for more control during the deployment process. Based on my user conversations, these profiles address many of said needs in an easy to use, straightforward manner. We are, of course, eager to know what you think. As always, you can let me know right here, through email, or on Twitter (@damrhein).