One of my favorite things to do with users or potential users of WebSphere CloudBurst is to help them understand how they can construct a custom environment using the appliance. Typically, we take one of their existing application environments and discuss the configuration steps that contribute to its makeup. From there, we map the required configuration actions to different customization capabilities in the appliance. It is one thing to talk about how you can customize every layer of your application stack with WebSphere CloudBurst, it is quite another to talk about it in the context of an existing environment. This exercise usually serves to greatly enhance a user's understanding of how to construct tailored environments with the appliance.
While I cannot take every one of you through this exercise in the context of one of your own application environments, I can propose a scenario that will help to illustrate the WebSphere CloudBurst customization process. Consider that I want to deploy a clustered WebSphere Application Server environment whose application server instances utilize WebSphere DataPower XC10 for HTTP session management. In order to deploy such an environment, I would need to do the following:
Install an OS and WAS
Install the WebSphere eXtreme Scale Client binaries - required for integration
Construct a clustered cell
Augment profiles with WebSphere eXtreme Scale profile templates
Configure the application server instances to use WebSphere DataPower XC10 for session management
So those are the steps, but how do they map to WebSphere CloudBurst? First, I know that the WebSphere Application Server Hypervisor Edition image used by WebSphere CloudBurst encapsulates the installation of the OS and WAS. I also know that WebSphere CloudBurst will automatically construct the clustered cell during the deployment process. That means I need to address the installation of client binaries, augmentation of profiles, and configuration of application server instances. In order to do this, I will use a combination of image extension and custom script packages.
To get started, I extend an existing WebSphere Application Server Hypervisor Edition image and simply install the WebSphere eXtreme Scale Client binaries. I then capture that image and store it as my own unique image in the WebSphere CloudBurst catalog. Now, you may wonder why I did not capture the profile augmentation in the custom image. Remember, you cannot change profile configuration during the extend and capture process as WebSphere CloudBurst resets the profiles as part of capturing the custom image.
My custom image encapsulates the installation of the client binaries, so now I turn to custom script packages. I need two in this case. One script package will augment a profile (either deployment manager or custom node) with the WebSphere eXtreme Scale profile template. The second script package will configure application server instances to use WebSphere DataPower XC10 for HTTP session management. Once done with these script packages, I have all the assets I need to build my target environment.
Using my custom image, I build a pattern that contains the number and kind of WebSphere Application Server nodes that I want. I use the advanced options to define a WebSphere Application Server cluster ensuring its creation happens during deployment. Next, I drag and drop the profile augmentation script onto the deployment manager and custom node parts in my pattern. Finally, I drag and drop the WebSphere DataPower XC10 configuration script onto the deployment manager. The pattern is now ready to deploy!
For those of you that are visual learners like me, this demonstration provides a nice overview of exactly what I wrote about above. Check it out and let me know what you think.
One of the things that often comes up at some point in IBM Workload Deployer conversations is the notion of self-service access. Specifically, users want to know what the appliance provides that enables them to allow various teams in their organization to directly deploy the middleware environments they need. In other words, they want to use IBM Workload Deployer to tear down the traditional barriers that exist between those that request the environment and those that fulfill said request. Now, as we begin to elaborate on this notion, it becomes quickly apparent that in order to effectively enable self-service, IBM Workload Deployer must deliver a few things.
First, IBM Workload Deployer must provide the means to define users with various levels of access. Second, IBM Workload Deployer must provide the means to define resource access at a fine-grained level to different users and groups of users. Check and check. The appliance has been doing this since the beginning of WebSphere CloudBurst. Without those two things, the conversation of self-service access would end pretty quickly. However, there is a final capability that is equally important: IBM Workload Deployer must deliver a means to limit resource consumption at a fine-grained level.
In IBM Workload Deployer there are a couple of ways to achieve this. First, you could define multiple cloud groups and allow access to those groups in a way that maps directly to resource entitlements. While that may work in some situations, others call for even more granularity. You may want to allow multiple different users or groups to access a cloud group, but you may want to allow different consumption limits for each of these groups. In this situation, you can take advantage of environment profiles and a new option when defining users of IBM Workload Deployer.
Consider the case that you have a group of developers and you want to limit their consumption of memory in the cloud. First, you start by defining your development users and for each you select Environment Profile Only as the value for the Deployment Options field.
By selecting the above value for the deployment options of a user, you restrict that user to only deploying via an environment profile as opposed to general cloud group deployments. After defining all of your development users, you may choose to organize them into a user group for easier management. At that point, you can define environment profiles and determine which ones your developers should have access to using the Access granted to field of the profile.
Within the environment profile, you can define resource consumption limits for compute resource and software licenses. For instance, you can define a limit on the amount of virtual memory consumed by all deployments using the profile. It is important to note that the limit is cumulative for ALL deployments that use the profile.
Now that all of the controls are in place, consider the deployment process for one of your development users. They pick a virtual system pattern, click the deploy icon and begin to configure the pattern for deployment. In the Choose Environment section of the deployment dialog, your development user will only be able to select the Choose profile option for deployment. Further, they will only be able to deploy using the environment profiles to which they have access.
After the deployment completes, a look at the Environment limits section in the profile shows the current usage totals.
Now suppose another development user, or even the same one, comes along and attempts to deploy another virtual system pattern even though the profile limits have already been reached. The user can initiate the deployment, but they will get a near immediate failure owing to the fact that they would exceed consumption limits if the deployment were allowed to proceed.
The same kind of enforcement occurs regardless of the resource limit type. You can use this approach to limit the consumption of CPU, virtual memory, storage, or software licenses among the various different users or groups of users you define in IBM Workload Deployer. If you combine fine-grained resource consumption limits with varying permissions and fine-grained access, I think you are on the road to truly enabling self-service in the enterprise.
Script packages are an integral part of virtual system patterns in IBM Workload Deployer. By attaching script packages to your patterns, you provide customizations particular to your unique cloud-based middleware environments. Customizations provided by script packages might include installing applications, creating application resources, integrating with external enterprise systems, and much more. The bottom line is, if you are creating virtual system patterns, you will almost certainly be creating script packages.
Largely, the act of creating a script package is independent of IBM Workload Deployer. The appliance does not dictate a particular scripting language, so all you need to do is make sure you can invoke your logic in the operating system environment. Your script package may be a wsadmin script, shell script, Java program, Perl script, and on and on. After you create the actual contents of your script package, you will then load that asset into the IBM Workload Deployer catalog.
Once loaded into the catalog, you define several attributes of your script package, including the executable command, command arguments, variables, execution time, and more. The process for defining these attributes is trivial using the intuitive UI in IBM Workload Deployer, but I wanted to take a little time to remind you of a technique I recommend to all users defining script packages. You can actually package a JSON file within the script package that defines all of the script's attributes. The format of the file is simple, and I am including an example below:
The example above is one taken from a script package in our samples gallery, and it shows the basics of which you need to be aware. Notice that in the JSON file, you can provide a name, description, unzip location, executable command, command arguments, variables, and more. You only need to ensure that the name of this JSON file is cbscript.json and that you include it at the root of the script package archive. Once you have done that, you load the script package archive into the catalog, refresh the script package details, and voila -- all the attribute definitions appear!
You may ask why I recommend this since it could seem like an unnecessary step. My answer to that is that you have to define these attributes anyway, so you might as well capture it once in the file. Once you capture it once in the file, you can ensure that if the same script needs to be reloaded, or if you need to move it to another appliance, its definition will be exactly the same (and presumably correct). I use this approach for all of my work, and for all of the samples I contribute to our gallery, and it really saves me a lot of misplaced effort that can result from typos. If you are out there creating script packages, try adopting this approach. I'm pretty sure you will be happy you did!
The concepts that govern users and user groups in WebSphere CloudBurst are fairly basic, but I get asked about them enough that I believe they warrant a short discussion. First things first, you can define users in WebSphere CloudBurst and optionally define user groups to assemble users into logical collections. For both users and user groups, you can assign roles that define the actions a particular user or group of users can take using the appliance.
All of that is straight forward, but it can get a bit tricky once we start considering the effects of user permissions when managing at the user group level. The basic premise is that when a user belongs to a group or groups, the user's effective permissions are a sum of the permissions to all of the groups to which they belong. While that is easy to say, and maybe even to understand, I feel like an example always helps.
Consider that we have a single user WCAGuy that belongs to the PatternAuthors, ContentCreators, and CloudAdmins groups. The permissions for those groups are as follows:
PatternAuthors: Users in this group have permission to create and deploy patterns
ContentCreators: Users in this group have permission to create catalog content as well as create and deploy patterns
CloudAdmins: Users in this group have permission to administer the cloud, create catalog content, and create and deploy patterns
Naturally then, it follows that the WCAGuy user can administer the cloud, create catalog content, create patterns, and deploy patterns. So then, what happens if we remove the WCAGuy user from the CloudAdmins user group? Well, as you may expect, there is an update to the user's permissions. The WCAUser user can no longer administer the cloud, but they can still create catalog content, create patterns, and deploy patterns (owing to their membership in the other two groups). Similarly, if we next removed the WCAGuy user from the ContentCreators group, then the user would retain only the permission to create and deploy patterns.
Just one more thing, let's talk about what happens when I remove a user from a group and they no longer belong to any groups. Consider that I created the WCAGuy user with the permission to create catalog content as well as create and deploy patterns. Next, I added the user to the CloudAdmins group, meaning the user now has the permission to administer the cloud. I promptly decide that the user has no business with those permissions, so I remove the user from the CloudAdmins group. What happens? The user retains the permission set of the last group to which they belonged. In this case, that means the WCAGuy user retains cloud administration rights. I have to update the user's permission set if I want to take that right away, but in this case, it will not automatically disappear upon removing them from the CloudAdmins group.
I hope this helps clear up any ambiguity you may have had concerning users, user groups, and permission sets in WebSphere CloudBurst.
IBM trekked further into the cloud today by announcing new offerings that will help clients to leverage the cloud within their enterprise. These offerings include development environments, test environments, and desktop services running either in the IBM cloud or a private cloud, as well as something called IBM CloudBurst (not to be confused with the WebSphere CloudBurst Appliance).
In particular, the IBM CloudBurst offering jumps out to me as a truly innovative offering. This offering promises our clients a complete private cloud solution that includes hardware, software, and services all in a single package.
Essentially it sounds like the IBM CloudBurst offering is all about building out a cloud-enabled data center. The hardware provides the bulk computing power to host private clouds, and the software provides enhanced service management capabilities to give users full control and insight over the elements of their private cloud.
Couple these computing capabilities with included IBM service, and this means clients should be able to get up and going with their private clouds very quickly.
This new offering really seems to hit a sweet spot. While the number of cloud computing offerings continues to grow, few if any offerings take such a holistic approach to providing such a solution.
This type of comprehensive solution gives users what they need in terms of the hardware to host a private cloud and the software to manage that cloud. Just as importantly though, it helps users put the hardware and software to work via implementation services also offered in the solution.
I invite you to take a look at the new IBM CloudBurst offering, and don’t forget to sign up for the IBM CloudBurst webcast scheduled for June 25th.
If you follow this blog often, you know that from time to time I like to post frequently asked questions. Well, it's been a while since I have done that, and since then I have added some new questions to my list -- along with some regulars. Take a look below, and if I don't answer your question feel free to leave a comment!
Can IBM Workload Deployer deploy software that is not IBM software? Yes. You can use one of the included images as a springboard and customize them with your own software via extend and capture. Additionally, you can use the IBM Image Construction and Composition Tool (I'm getting ahead of myself here) to create your own custom images from the ground up and use those within IBM Workload Deployer.
Can I use VMotion for the systems I deploy with IBM Workload Deployer? Yes. IBM Workload Deployer has tolerated the use of VMotion since the WebSphere CloudBurst days (see the Additional Considerations section on this page for more information). IBM Workload Deployer v3 introduced the notion of virtual machine mobility initiated directly from the appliance. This capability takes advantage of VMotion in the case of VMware-based cloud environments.
Can IBM Workload Deployer deploy just a base operating system? Yes. IBM Workload Deployer v3 introduced a base operating system image that contains 64-bit Red Hat Enterprise Linux. Internally, IBM Workload Deployer uses this as the foundation on top of which virtual application patterns are deployed. You can use it to deploy virtual machines containing just the base OS, or you can customize it to deploy software of your choosing. (As an aside, IBM Workload Deployer v3.1 will include a base operating system image for AIX)
Can I automate the process of calling/using IBM Workload Deployer? Yes. IBM Workload Deployer is built to fit a specific need -- creating and managing a cloud of middleware and middleware-based workloads. In that light, it would be a shortcoming if IBM Workload Deployer did not to fit well into more holistic or enterprise-wide cloud management systems. The REST API and CLI allow you to automate the use of IBM Workload Deployer, thereby allowing it to be mashed up into other processes.
Can I group two appliances together for high availability? Yes. IBM Workload Deployer v3.1 introduces the ability to configure appliances in a master/slave setup. You can connect two appliances, allow them to share a floating IP address, and be confident that data is continuously replicated between the two. If one appliance fails, the other appliance picks up the floating IP ensuring continuous service.
Are images created using the Image Construction and Composition Tool supported for use within IBM Workload Deployer? Yes. Part of the new IBM Workload Deployer 3.1 announcement was a statement of support for using images created by the Image Construction and Composition Tool as a component of your virtual system patterns. This is a very important enhancement as it allows you to extend the set of content deployed by IBM Workload Deployer while being sure that you are operating within the boundaries of intended use.
Can I use IBM Workload Deployer to provision to public clouds? No... and yes. If you install an IBM Workload Deployer appliance in your datacenter, you cannot use it to deploy to a public cloud environment. However, you may have recently heard about the IBM SmartCloud Application Services portfolio. IBM has announced that the pattern-based provisioning that one gets with IBM Workload Deployer will also be available as part of this portfolio. This means that you will be able to build and deploy patterns using a service hosted on the IBM SmartCloud. Further, your deployed systems will run on the IBM SmartCloud. Check out this demo for more information.
** IBM Workload Deployer 3.1 firmware is available on 11/18.
I want to clear something up about WebSphere CloudBurst that can sometimes cause a bit of confusion. In nearly all of our content about the appliance, we talk about it in the context of building private clouds consisting of WebSphere application environments. Typically people think of private clouds as something only those within their organization can access and utilize. However, with WebSphere CloudBurst you are not limited to creating that kind of a private cloud.
Perhaps it is more fitting that we talk about WebSphere CloudBurst as a means to create on-premise clouds. After all, that's really what we mean. You create a shared pool of hardware and network resources owned by your organization, and then you define this cloud of resources to WebSphere CloudBurst. Once that cloud is defined, you can leverage WebSphere CloudBurst to dispense your WebSphere application environments into that cloud. The accessibility of your application environments running in that cloud is entirely up to you.
You may decide that the cloud is indeed private and that only those in your organization or a smaller subset of users can access the environments. On the other hand, you may decide that you want to allow consumers in the public domain to request WebSphere application environments and then have WebSphere CloudBurst provision those environments into a public cloud. I say public here because while the cloud's resources are on your premise, access to that cloud is not restricted to within the organizational firewall. Ultimately, the determining factor for whether or not your WebSphere CloudBurst cloud is public or private is the network configuration you provide. If the virtual machines are associated with network resources that are publicly accessible, then I would say you have a public cloud.
I hope this entry didn't serve to only add to the confusion. The bottom line is this: WebSphere CloudBurst allows you to create, deploy, and maintain virtualized WebSphere environments in an on-premise cloud. Whether that cloud is public or private is entirely up to the network configuration that you setup.
As a final preview of this week's building block sessions in the Enabling cloud computing with WebSphere campaign, I caught up with WebSphere DataPower architect Tim Smith. Tim is delivering a podcast that introduces and explains the new Application Optimization capabilities in the WebSphere DataPower line of products. Here is what Tim had to say:
Me: I speak with quite a few customers about the WebSphere CloudBurst Appliance, and for once I'm happy to be the one asking this question. Why do we deliver WebSphere DataPower in the appliance form factor?
Tim: DataPower has become a dominant player in the DMZ and in the ESB. Much of the reason is that this is a purpose built hardware appliance. There are many things that our customers like about this appliance package. First, it has security as part of its DNA. The basis for securing connections, applies throughout the network whether it is in a DMZ or in an ESB. The physical box provides tamper resistant protection. Another reason is availability -- there are no spinning media, dual power supplies, and a focus on fail over support.
In both the DMZ and the ESB, there has been a proliferation of products. The main reason for the proliferation is that customers want to remove as many decisions from the general purpose server as possible, and let servers do what they do best, process application requests. The devices that have been proliferating make more decisions on the request. They do deep packet processing and routing. They also may transform the request into an entirely different request. So, there are an abundance of "pre-processing" decisions and operations made. With DataPower, many functions are integrated into the single hardware platform, giving you a smaller box count. No need to purchase and maintain several platforms, their OS and software versions, compatibility lists, etc. With a single hardware box that does so many things, we can greatly reduce the total cost of ownership for our users.
The DataPower appliance is a blend of Hardware and firmware that is well provisioned with hardware assists that help compile, parse, and assist in many of the intensive packet processing capabilities. To summarize, you get an extremely flexible and adaptable product that reduces total cost while increasing performance.
Me: A theme that comes up in cloud computing over and over is consolidation. Can you speak to the consolidation offered by WebSphere DataPower appliances with respect to the self-balancing capabilities?
Tim: Yes. My answer to the prior question was a long-winded way of describing DataPower's ability to consolidate many features into a single platform. Self-balancing is an example. As DataPower became more popular, larger installations required multiple DataPower appliances in a tier of platforms. A common architecture was to place a load balancer or IP sprayer in front of the tier to distribute the traffic evenly among the tier of DataPower appliances. An IP sprayer is an example of another platform that needs to be added to the environment. It is another box that must be purchased, managed, and maintained. Self-balancing is a feature that was added to DataPower to eliminate the need for an IP sprayer. The way it works is that one of the DataPower appliances in the tier owns the Virtual IP (VIP) Address. It receives all of the traffic, and then distributes it to each of the other DataPower appliances in the tier. If the DataPower appliance that owns the VIP address goes down, one of the others is elected and it takes over. The result is one less product required to support the same level of functionality.
Me: For much of the past, cloud computing mostly focused on virtualization and management of resources at the raw compute level (servers, storage, networking, etc.). While there is definitely ongoing focus here, we start to see it moving up the stack towards applications, and part of that effort includes more evolved application load distribution. With that in mind, how can WebSphere DataPower help users more effectively distribute requests to their applications?
Tim: If a front end appliance or gateway device can dynamically learn information about its environment, specifically the back end, it will be able to make better decisions on how and where to route the request. This is one of the tasks that the Application Optimization feature addresses. Information from the back end can of course be manually configured, but the real value in cloud computing is dynamically adapting when new server resources are brought on line or are taken off line. In the 3.8.0 release, we implemented something called Intelligent Load Distribution (ILD). Intelligent load distribution focuses on continually learning the topology of a back end, updating DataPower's load balancers with that information, and distributing the load based on the updates. In addition to the topology, ILD learns the weights associated with each server. These weights can continually and automatically change as traffic patterns change. The result is load balancing to the back end that sends the optimal amount of load to each server.
Another traffic distribution aspect incorporated into ILD is session affinity. When a server application needs to receive every request from a given client, session affinity is used to route the requests to the same server. In some sense, session affinity overrides the load balancing algorithm. The session affinity support works with any type of back end server, but with a WebSphere back end, all session affinity information is automatically configured.
Me: Continuing on the theme of application intelligence, what is this new Application Routing option in WebSphere DataPower?
Tim: ILD focused on learning the topology of the network and making better decisions based on an ever changing cloud topology. Application Routing does something similar by learning which applications are running on each server. Once a request is handed to DataPower's load balancer, the request is classified as to the application that it is targeted for. Then the request is load balanced amongst the servers that are running that application. The information to perform application routing is dynamically learned and changes as applications are added or removed.
WebSphere has invested substantially in managing the life cycle of an application. Changing from one edition of an application to the next sounds like an easy task, but it can be very difficult to perform this type of maintenance on a production environment. The DataPower appliance supports life cycle management by working with the WebSphere back end to provide group and atomic edition rollout. The rollout feature allows traffic to be gracefully diverted from servers that are being taken offline and reloaded with the new application edition. This rollout can be done while leaving the other applications on the server unaffected. This support makes edition rollout a very simple task for the system administrator.
Are you planning on attending IBM IMPACT? For all of you WebSphere users out there, I sure hope the answer to that is yes. Simply put, there is no better event to attend in order to learn about everything going on across the IBM WebSphere portfolio. There is already an exciting array of technical sessions lined up that will touch on topics such as analytics, big data, cloud computing, elastic data caching, enhanced automation, business process management, hybrid clouds, and much more. In addition to the sessions, you will also get plenty of chances to work directly with various WebSphere solutions in our labs. For my fellow hands-on learners, this is a great opportunity.
We have technical sessions and labs every year at IMPACT though, so you may be wondering besides the content, what's new?? Well, in that regard I'm excited to point out that this year's IBM IMPACT will host the first ever WebSphere Unconference! For those of you unfamiliar with the unconference format, the basic idea is that a group of folks with common interests (cloud computing, big data, etc.) get together and drive a series of conversations/sessions around these topics of interest. The unconference will include lightning talks (5 minutes to make a point!), guest appearances, as well as user-driven breakout sessions. These events are a great way to network with fellow WebSphere users, discuss cool technology, explore common challenges, and otherwise expand your areas of interest and knowledge.
The official website of the WebSphere Unconference recently launched in order to support this event. Besides giving you some details about the unconference, the website allows you to submit your own ideas for breakout sessions. All you need to do is sign up on the site, and you can start to put forth your own topic ideas. You can vote for the different sessions that interest you, and the top vote getters will earn a spot during the unconference.
I encourage you to go and have a look at the WebSphere Unconference website. It is easy to sign up and easy to use. If you are heading to IMPACT, please make time to at least drop by the unconference on Thursday. I promise you will benefit from engaging, attendee-led conversation and discussion. I hope to see you there!
When I first started to become aware of the cloud computing movement, I remember being intrigued but not all that aware of its possible consequences to me. After all, I was a developer not a systems administrator, so other than professional curiosity why was cloud computing all that important to me? Maybe you are a developer that can see right through my early, naive perception of cloud computing, but maybe you are a developer that, like me in the early going, doesn't quite see why cloud computing should matter to you. In the case of the latter, I've come to realize that there are several reasons why cloud computing matters to the developer. Let me try to sum up a few of those reasons for you here.
Reason #1: Developer services can be delivered via the cloudThere are many different types of services that can be realized from a cloud (public, private, or hybrid) that could have a large impact on the way developers work. As I mention in a previous post, IBM announced a Tools as a Service initiative in which IDEs are made available within a public cloud. IDEs in the cloud give developers a single development environment that can be accessed from any machine at any time. Better yet, we don't have to worry with installing and maintaining the environment. In addition to IDEs in the cloud, with the increased focus on virtualization and virtualization management that cloud is bringing, the ability to rapidly procure and instantiate runtime environments should become standard practice. This means that new ideas and new product code can be rapidly prototyped and tested. No longer should a proof of concept be delayed because it couldn't be proven in a runtime environment.
Reason #2: Cloud computing means a world of new products and offeringsAs a developer, it is a continuous battle to keep up with constantly emerging technologies, but it is imperative that we do so in order to ensure we take full advantage of available solutions. Cloud computing providers introduce a whole new world of service offerings for consumption by application developers. Cloud providers are offering new storage solutions, new database implementations, new content distribution mechanisms, new application integration capabilities, etc. As developers who may potentially be writing applications that run in the cloud, these new offerings directly affect the code we write. We need to educate ourselves about these new services, and we should understand when these solutions can be best leveraged to deliver our end product.
Reason #3: SOA becomes more importantOkay, so maybe this is not aimed squarely at the developer, but I know many times a developer wears the hat of architect as well... even if they don't know it! In a cloud computing world, the applications and services we deploy to the cloud should align and fit into our SOA. This is critical if we are to fully exploit the benefits of ubiquity offered by the cloud. Cloud computing inherently provides the ability to access services from any machine with a network connection, automatically giving the kind of service ubiquity sought by many companies. By developing these services in a SOA-compliant manner, we extend the reach by making it more readily consumable by other application components. We move beyond pure end-user applications and services, and in doing so new or increased revenue streams may be realized for the service.
These are just a few of the ways I see cloud computing currently affecting the developer's role. There are a myriad of reasons that developers should be cognizant of cloud computing, and I expect the list of reasons to boom as cloud computing continues to advance. I'd also like to hear what you think about cloud computing and the developer, so post a comment below if you would like to join the discussion.