Reducing deployment time, increasing consistency, and fostering agility are benefits you would likely expect when exploring cloud-based approaches for your middleware application environments. Traditional approaches to setting up these kinds of environments have typically been time- and resource-intensive, often requiring several weeks and multiple people to construct.

Further, achieving consistent provisioning results is often difficult due to the number of steps and complexity involved. Besides contributing to the increasing costs in IT over the last 10 years, these challenges prevent companies from achieving the kind of agility they need in the rapidly shifting consumer markets of our time.

Enter IBM® Workload Deployer. This innovative solution, based on its predecessor, the WebSphere® CloudBurst Appliance, tackles these traditional issues making the deployment of cloud middleware environments fast, repeatable, and efficient.

A pattern-based approach is the foundation of IBM Workload Deployer and its benefits. Using the cloud management appliance, you build and deploy patterns that represent your completely configured application environments. These patterns, sourced by virtual images, allow you to represent your complete, often complex, environments as a single deployable unit.

When you are ready to use a particular application environment, you simply pick a pattern and deploy it. IBM Workload Deployer automates the deployment, configuration, and integration of the various virtual machines that make up your environment and it delivers the completed product in a matter of minutes.

Besides obvious benefits of unprecedented speed, consistent patterns also guarantee consistent results from deployment to deployment, freeing you up to focus more time on your applications and less time ensuring the correct configuration of the supporting environment.

To foster rapid adoption, IBM Workload Deployer ships with a pre-built set of virtual images, virtual system patterns, and virtual application patterns. You can immediately deploy these and benefit from rapid and consistent delivery of general purpose middleware application environments.

However, creating your own custom patterns will further enhance the value you receive from this approach. In that regard, IBM Workload Deployer provides comprehensive customization techniques for both pattern models it supports.

IBM Workload Deployer inherits virtual system patterns from its predecessor, WebSphere CloudBurst. While the product name has changed, the general concepts and techniques for customizing this type of pattern remain pretty well intact. This article does not focus on those customization techniques, but you can consult the developerWorks series Customizing with WebSphere CloudBurst for an in-depth look at the various customization approaches for this type of pattern.

The remainder of this article is focused on the customization of virtual application patterns in IBM Workload Deployer and concludes with an example of one such customization.

Customizing virtual application patterns

Virtual application patterns are new deployment models introduced by IBM Workload Deployer version 3. This type of pattern takes a decidedly application-centric approach for building, deploying, and managing middleware application environments in a cloud.

You build a virtual application pattern by supplying your application and specifying both functional and non-functional requirements for that application. IBM Workload Deployer takes that input from you and transforms it into a completely installed, configured, and integrated middleware application environment as depicted in Figure 1.

As a user of virtual application patterns, you do not need to know how to install, configure, or integrate the middleware and applications because the pattern encapsulates that knowledge. How do the patterns do this? It's all a matter of plug-ins.

The role of plug-ins for virtual application patterns

The basic example of an IBM Workload Deployer Pattern for web applications shown in Figure 2 highlights the major elements of a virtual application pattern.

As you can see, virtual application patterns consist of components, links, and policies:

• Components represent functional profiles for a given workload. In the case of the virtual application pattern for web applications, there are components for enterprise applications (EAR files), web applications (WAR files), databases, OSGI applications, and more. These components provide the essential pieces of the environment you are attempting to build and deploy.
  
• Links express an integration point between components in a virtual application pattern. In the pattern shown in Figure 2, a link specifies that the enterprise application has a dependency on the database component in the pattern.
  
• Policies like the Scaling Policy shown in Figure 2 allow you to specify functional and non-functional requirements for your application environment. Policies dictate both configuration and ongoing management aspects of the environment.

When you build and deploy virtual application patterns, you interact with a visual editor in the form depicted by Figure 2. From an end-user perspective, this makes it easy to quickly construct and deploy fully configured, integrated, and dynamic cloud-based environments.

Something has to provide the function implied and encapsulated by virtual application patterns — that is where plug-ins enter the picture. All of the installation, configuration, integration, and management functionality that you get from a virtual application pattern comes from a plug-in or collection of plug-ins. Plug-ins define the components, links, and policies available to users of a virtual application pattern and back those elements up with necessary functionality.

The good news is IBM Workload Deployer has an open design that allows you to contribute your own plug-ins to the system. Why is this important? If you want to create your own virtual application pattern type in order to service a custom workload on custom software or if you want to extend one of the virtual application patterns that IBM Workload Deployer ships, you need to develop one or more custom plug-ins.

The focus of this article is creating a custom plug-in that extends the IBM Workload Deployer Pattern for web applications with WebSphere® Application Server Community Edition support. This example provides background on the overall architecture of plug-ins, the use of the new Plugin Development Kit from IBM, and the means with which you can add your custom plug-ins to IBM Workload Deployer. Let's get started!

Back to top

Makeup of a plug-in

Before jumping into creating a custom plug-in to support WebSphere Application Server Community Edition, you need to understand the makeup of a plug-in. Six configuration points make up typical plug-in:

1. Before deployment, the overall definition of the plug-in is defined in the config.json file.
2. At deployment, the application model is transformed into an unresolved topology.
3. The unresolved topology is transformed into a resolved topology.
4. Required resources are provisioned and the resolved topology is transformed into a final topology.
5. The actual deployment is deployed into the private cloud.
6. The software is installed, configured, and started on the deployed virtual machines.

Pre-deployment

The overall definition of a plug-in is defined by the config.json configuration file. The config.json file contains information about:

• Packages (middleware or other software) to be installed.
• Basic hardware requirements (for example, 32 bits versus 64 bits of VM hosting installed package).
• Plug-in name, version, and associated pattern type.

The Virtual Application Builder tool is used to build out your virtual application pattern. The pattern is made up of components, links, and policies, each of which is defined in a plug-in configuration file called metadata.json. The metadata.json file defines the following for each component, link and policy:

• Name, description, and icon used to graphically represent the plug-in in Virtual Application Builder.
• Configurable attributes which are later used during configuration of the package on the deployment VM.

Deployment

The Virtual Application Builder takes the pattern that you created using information defined in the metadata.json and creates an application model; Figure 2 is an example of a graphical representation of an application model that is composed of components, links, and policies. The application model then is progressively elaborated upon until finally deployment occurs. IBM Workload Deployer orchestrates the execution of each of these steps.

The first step in the deployment process is to convert the application model into an unresolved topology. The unresolved topology is generic and includes information such as what package to include and requirements for memory and disk. This conversion makes use of the <template>.vm that you define.

The second step is to take the unresolved topology and convert it into a resolved topology. The config.json file contains the necessary transform data to complete this process. A key part of this process is to map the requirements of the application model (operating system, architecture, disk, memory) to parts provided by plug-ins.

The third step is to create a final topology. IBM Workload Deployer takes the resolved topology document, provisions any required resources, and writes the final topology document.

The fourth step is the actual deployment into the private cloud. The deployment process and corresponding configuration file required for the step are outlined in Figure 3.

The fifth and final step is to install, configure, and start the software on the deployed virtual machines. This is accomplished with a series of life cycle scripts that you define for each software component to be installed: install.py, configure.py, and start.py.

Parts and nodeparts

Before diving into the custom plug-in creation part of this article it is worth talking about parts and nodeparts since they figure heavily in the following sections.

Packages in plug-ins can contain both parts and nodeparts. Each part and nodepart corresponds to a particular artifact (such as a software product). From a technical perspective, parts and nodeparts are really the same thing in that they both are used to install, configure, and start software.

From a logical perspective nodeparts are generally used to install, configure, and start system level software such as a firewall. Parts are used to install, configure, and start middleware-type software such as WebSphere Application Server Community Edition.

Back to top

Creating a custom plug-in

The rest of the article details the process of creating a custom plug-in to provide support for enterprise application (EAR) and web application (WAR) files running on WebSphere Application Server Community Edition (WAS CE). As a general overview, the following steps cover:

1. Defining and packaging plug-in artifacts.
2. Defining configurable application model components to be exposed in the Virtual Application Builder.
3. Defining template(s) to convert the application's visual model into a physical model.
4. Defining life cycle scripts to install, configure, and start software on deployed virtual machine.

Define and package plug-in artifacts

The available options for packaging the artifacts of a plug-in shortly are described, but no matter what packaging mechanism you decide to use, you have to specify information about those artifacts in the config.json file within the plug-in's archive. Listing 1 shows the config.json for our WASCE plug-in.

{
   "name"    : "wasce",
   "version" : "1.0.0.1",
   "patterntypes": {
      "secondary"  : { "*":"*" }
   },
   "packages" : {
      "WASCE" : [ {
            "requires"  : {
               "arch"   : "x86_64",
               "memory" :  512,
               "disk"   :  300
            },
            "parts":[ {
                  "part"  : "parts/wasce.tgz",
                  "parms" : {
                     "installDir" : "/opt/wasce"
                  }
            } ], 
      "WASCE_SCRIPTS":[ {
            "parts":[ {
                  "part":"parts/wasce.scripts.tgz"
            } ]
      } ]
      } ]
   }
}

The config.json file is the only required file in a plug-in. The name, version, and patterntypes elements are all required:

• The name element specifies the name of the plug-in.
• The version element defines the version number of the plug-in.
• The patterntypes element specifies the pattern types with which the plug-in is associated.

Our WASCE plug-in extends the IBM Workload Deployer for Web Applications pattern by specifying webapp as the patterntypes value. This means the components you are defining will be available when creating patterns from the web application pattern shipped with IBM Workload Deployer.

The requires element provides a way for you to specify required operating system and bit architecture configuration for the virtual machines deployed for your plug-in. For our plug-in we specify a 64-bit, x86 architecture for the virtual machines.

The memory and disk elements specify the minimum disk and memory requirements for each package defined by your plug-in. During the provisioning process, IBM Workload Deployer will add up the minimum values for each package and provision a virtual machine that meets the specified requirements.

The packages element defines the file packages with both the part and nodepart elements. Our plug-in provides two packages: WASCE and WASCE_SCRIPTS. The WASCE package contains the parts/wasce.tgz part file. This archive contains the binaries required to install WebSphere Application Server Community Edition and we package it directly in the plug-in.

There are other options for specifying the required binaries. You can define a file attribute and have administrators upload the required binaries after loading the plug-in in IBM Workload Deployer. You can also link to a remote server that stores the required artifacts. The WASCE_SCRIPTS package provides the life cycle scripts to install our WASCE image to the desired location, to install the EAR or WAR file, and to start the server.

Define configurable application model components

Ignoring the creation of the icon graphic, this step is easy. The web and enterprise applications are components that you want to show to users in the Virtual Application Builder. As such, specify each of them in the metadata.json file that is located in the plugin/appmodel directory of the plug-in archive. Listing 2 shows the JSON to define the web archive component.

[
 {
        "id"          : "WARCE",
        "label"       : "Web Application (WebSphere Application Server 
                         Community Edition)",
        "description" : "A web application cloud component represents an execution 
                        service for Java EE Web applications (WAR files).",
        "type"        : "component",
        "thumbnail"   : "appmodel/images/WASCE.png",
        "image"       : "appmodel/images/WASCE.png",
        "category"    : "application",
        "attributes"  : [
            {
                "id"          : "archive",
                "label"       : "WAR File",
                "description" : "Specifies the web application (*.war) to 
                                be uploaded.",
                "type"        : "file",
                "required"    : true,
                "extensions"  : [ "war" ] 
            }
        ]
 } 
]

There is a similar stanza for the enterprise archive component for its downloadable archive. The first type field of the listing is important. The value options for this field are component, link, or policy and this defines the type in the application model. The id of our component is WARCE. This can be any value as long as it is unique. The category refers to the tab under which this component is shown on the palette in the Virtual Application Builder. The attributes array defines properties for the component you are defining. Users will see and be able to specify values for these properties when using this component in the Virtual Application Builder. Attribute types include file, string (shown here), number, Boolean, array, and range.

Define template to convert visual model into physical model

Plug-ins must provide the knowledge for how to implement, or realize, the deployment of the defined components In our example, we need to specify what it means to deploy an Enterprise or Web Application component.

To do this, we provide a single transform that translates the application model derived from what users build in the Virtual Application Builder into a concrete topology. Listing 3 shows a fragment of a JSON file used for the transformation. Each component and link must have a transform. In our plug-in, our WARCE and EARCE components share the same transform template.

{
   "vm-templates":[
      {
         "name"     : "${prefix}-wasce",
         "packages" : [ "WASCE", "WASCE_SCRIPTS" ],
         "roles"    : [
            {
               "plugin"       : "$provider.PluginScope",
               "name"         : "WASCE",
               "type"         : "WASCE",
               "quorum"       : 1,
               "external-uri" : [{"ENDPOINT":"http://{SERVER}:8080"}],
               "parms":{
                  "ARCHIVE"   : "$provider.generateArtifactPath( $applicationUrl, 
                                ${attributes.archive} )"
               },
               "requires"     : { "memory":512, "disk":300 }
            }
         ],
         "scaling" : { "min":1, "max":1 }
      }
   ]
}

The topology fragment is a JSON object that contains a vm-templates element which is an array of vm-templates. A vm-template is a virtual machine template; it defines the parts, nodeparts, and attributes of a virtual machine to be deployed. For this example, we only need a single vm-template that contains four important elements:

1. name: A unique name for a deployed virtual machine.
2. packages: This specifies a list of parts and nodeparts installed on each deployed virtual machine. The WASCE entry indicates the use of our WASCE virtual image, and the WASCE_SCRIPTS entry specifies the WASCE life cycle scripts (which we'll discuss next).
3. roles: parts in a plug-in invoke life cycle scripts for roles. You can have one or more roles in your plug-in, but in our case we have a single WASCE role. When all roles on a node go to the RUNNING state, the node itself goes to the green RUNNING state.

Define life-cycle scripts to install, configure and start software

Now you are ready to define the life cycle scripts for your plug-in. In total, write scripts to install, configure, and start the components of our plug-in. You can view the complete scripts in the downloadable archives, but some of the key artifacts are highlighted here.

wasce install.py

The install.py script copies our WASCE image from the download location to the desired installDir. It also sets the installDir value in the environment for subsequent scripts.

All parts and nodeparts installed by the IBM Workload Deployer agent run as root. The chown -R virtuser:virtuser command changes file ownership of the installed contents to the desired user and group.

Finally, the install.py script makes the scripts in the WebSphere Application Server Community Edition's bin directory executable. Listing 4 shows the contents of the install.py script.

installDir = maestro.parms['installDir']
maestro.trace_call(logger, ['mkdir', installDir])

if not 'WASCE' in maestro.node['parts']:
    maestro.node['parts']['WASCE'] = {} 
maestro.node['parts']['WASCE']['installDir'] = installDir

# copy files to installDir to install WASCE
this_file = inspect.currentframe().f_code.co_filename
this_dir = os.path.dirname(this_file)
rc = maestro.trace_call(logger, 'cp -r %s/files/* %s' % (this_dir, installDir), 
 shell=True)
maestro.check_status(rc, 'wasce cp install error')

rc = maestro.trace_call(logger, ['chown', '-R', 'virtuser:virtuser', installDir])
maestro.check_status(rc, 'wasce chown install error')

# make shell scripts executable
rc = maestro.trace_call(logger, 'chmod +x %s/bin/*.sh' % installDir, shell=True)
maestro.check_status(rc, 'wasce chmod install error')

Listing 4 shows how the script makes use of the maestro module provided within the plug-in framework. This module provides several helper methods that are useful during installation and elsewhere.

Wasce.scripts install.py

The wasce.scripts part also contains an install.py script. This script installs our WebSphere Application Server Community Edition life cycle scripts.

# Prepare (chmod +x, dos2unix) and copy scripts to the agent scriptdir
maestro.install_scripts('scripts')

wasce.scripts configure.py

The configure.py script in the wasce.scripts part installs the user-provided application to WebSphere Application Server Community Edition. The script takes advantage of the hot deploy capability of WebSphere Application Server Community Edition and simply copies the application binaries to a monitored directory. Listing 6 shows the contents of the configure.py script.

installDir = maestro.node['parts']['WASCE']['installDir']

ARCHIVE = maestro.parms['ARCHIVE']
archiveBaseName = ARCHIVE.rsplit('/')[-1]
# Use hot deploy
deployDir = os.path.join(installDir, 'deploy')
if os.path.exists(deployDir) == False:
        # Make directories
        os.makedirs(deployDir) 
deployFile = os.path.join(deployDir, archiveBaseName)

# Download WASCE archive file
maestro.download(ARCHIVE, deployFile)

wasce.scripts start.py

The start.py script in the wasce.scripts part is responsible for starting the WebSphere Application Server Community Edition process. After starting the process, the script updates the state of the role to RUNNING. Once the deployment is in the RUNNING state, you can access the deployed application environment. Listing 7 shows the use of the geronimo.sh start command to start WebSphere Application Server Community Edition as well as the use of the gsh.sh command to wait on startup.

wait_file = os.path.join(maestro.node['scriptdir'], 'WASCE', 'wait-for-server.txt')

installDir = maestro.node['parts']['WASCE']['installDir']

rc = maestro.trace_call(logger, ['su', '-l', 'virtuser', installDir + 
  '/bin/geronimo.sh', 'start'])
maestro.check_status(rc, 'WASCE start error')

logger.info('wait for WASCE server to start')

rc = maestro.trace_call(logger, ['su', '-l', 'virtuser', installDir + '/bin/gsh.sh', 
  'source', wait_file])
maestro.check_status(rc, 'wait for WASCE server to start error')

maestro.role_status = 'RUNNING'

logger.info('set WASCE role status to RUNNING')  

logger.debug('Setup and start iptables')
maestro.firewall.open_tcpin(dport=1099)
maestro.firewall.open_tcpin(dport=8080)
maestro.firewall.open_tcpin(dport=8443)

There are other scripts and artifacts that make up the plug-in, but these are the most significant ones. You can explore the complete contents of the plug-in by downloading the plug-ins at the bottom of the article.

Eclipse plug-in summary

Figure 4 shows all the parts of our wasce plug-in as seen from the Package Explorer view in Eclipse.

Back to top

Installing and using the plug-in

Building the WebSphere Application Server Community Edition plug-in packages its contents in a .tgz file. Once packaged, you are ready to load the new plug-in into the IBM Workload Deployer repository. After loading, you can use the components in the new plug-in to build and deploy a virtual application pattern. (See IBM Workload Plugin Development Kit in the Resources section for instructions on getting and using the free Workload Plugin Development Kit to perform this step.)

Loading plug-ins

To load the plug-in:

1. Open the IBM Workload Deployer user interface.
2. From the menus on the top of the page, select the Cloud menu and click the System Plug-ins link.
3. Click the green + icon and upload the .tgz file. After loading the file, you can view the attributes of the plug-in as shown in Figure 5.
   
   Figure 5. The imported plug-in definition
   
   
   

As you can see in Figure 5, the wasce plug-in shows both the Enterprise Application and Web Application components that you defined.

Using the new plug-in

Now that you are done loading the new plug-in, you can use its components to build a new virtual application pattern. To do that, use the Virtual Application Builder in IBM Workload Deployer.

1. Click the Patterns menu on the top toolbar and then click the Virtual Applications link.
2. Click the green + icon to create a new pattern. As shown in Figure 6, make sure the pattern type is Web Application and select the blank template.
   
   Figure 6. Creating a new virtual application pattern
   
   
   
3. Click the Start Building button to enter the Virtual Application Builder.
4. Once in the builder, expand the Application Components section. You should see both Enterprise Application and Web Application components based on the WebSphere Application Server Community Edition server. You can drag and drop those components to the canvas to add them to your pattern. Figure 7 shows the use of the new Web Application component.
   
   Figure 7. Using the new components
   
   
   
5. Per the definition of the component in the plug-in, there are both Name and WAR File properties. Both are required and the WAR File property contains a file upload dialog. After you complete the construction of your new pattern, save it by clicking the Save or Save As button at the top of the Virtual Application Builder. At this point, you are ready to deploy the new pattern. (For more on this, see the wasce samples installation bundle download in Resources.)

Deploying the custom virtual application pattern

1. Go back to the virtual application pattern page by selecting the Patterns menu and clicking on the Virtual Applications link.
2. Select your new virtual application pattern from the list on the left side of the screen.
3. Click the Deploy to Cloud icon, select the appropriate cloud group, and then click okay.

IBM Workload Deployer begins the deployment of the environment and it automatically invokes the life cycle scripts you provided in the plug-in to install, configures, and starts the environment.

Viewing the deployed application

To monitor the deployment process, select the Instances menu and then click the Virtual Applications link. Each virtual machine in the deployment has an associated status. For a web application or enterprise application virtual machine, an endpoint link should appear once the status of the deployment reaches the RUNNING state as shown in Figure 8.

If you click the Endpoint link, a pop-up dialog displays the URL for your deployed application. Click that URL to access your application.

Back to top

In conclusion

Virtual application patterns represent a bold step toward a more application-centric cloud experience. Like virtual system patterns, the design of IBM Workload Deployer allows for extensive customization of this deployment artifact. By creating plug-ins, you can enable the creation of highly customized virtual application patterns, thereby creating cloud-based deployments suited for your needs. Stay tuned for further articles in this series that expand on plug-in and virtual application pattern customization techniques!

Resources

Learn

• Learn more about the IBM Workload Plugin Development Kit from the wiki.
  
  
• Join the IBM Workload Deployer Community at developerWorks to stay informed.
  
  
• For more on how to perform tasks in the IBM Cloud, visit these resources:
  • Up and download files from a Windows instance.
  • Install IIS web server on Windows 2008 R2.
  • Create an IBM Cloud instance with the Linux command line.
  • Create an IBM Cloud instance with the Windows command line.
  • Extend your corporate network with the IBM Cloud.
  • High availability apps in the IBM Cloud.
  • Parameterize cloud images for custom instances on the fly.
  • Windows-targeted approaches to IBM Cloud provisioning.
  • Deploy products using rapid deployment service.
  • Integrate your authentication policy using a proxy.
  • Configure the Linux Logical Volume Manager.
  • Deploy a complex topology using a deployment utility tool.
  • Provision and configure an instance that spans a public and private VLAN.
  
  
• In the developerWorks cloud developer resources, discover and share knowledge and experience of application and services developers building their projects for cloud deployment.
  
  
• Find out how to access IBM SmartCloud Enterprise.
  
  

Get products and technologies

• Download the IBM Plugin Development Kit with build libraries and build scripts to build plugins and pattern types, sample plugins and pattern types, and a user's guide.
  
  
• You can download the wasce_samples-version.zip sample installation bundle here.
  
  
• See the product images available for IBM SmartCloud Enterprise.
  
  

Discuss

• Use the Message Board of the IWD developerWorks Community to ask questions.
  
  
• Join a cloud computing group on developerWorks.
  
  
• Read all the great cloud blogs on developerWorks.
  
  
• Join the developerWorks community, a professional network and unified set of community tools for connecting, sharing, and collaborating.
  
  

About the authors

Dustin Amrhein joined IBM as a member of the development team for the WebSphere Application Server. While in that position, Dustin worked primarily on web services infrastructure and web services programming models. In addition, Dustin worked on the development of a RESTful services framework for Java runtimes. In his current role Dustin is a WebSphere Client Technical Professional.

Ted Kirby develops plug-in support for IBM Workload Deployer Pattern for Web Applications at IBM in the Research Triangle Park, North Carolina. He is an Apache Geronimo committer and was a WebSphere Application Server Community Edition developer. Previously, he was a WebSphere Technical Evangelist for Extreme Transaction Processing and has enhanced and maintained eCommerce websites and developed distributed operating systems, including the system used by the Deep Blue machine.

Brian Stelzer is a Software Engineer and Team Leader on the IBM Workload Deployer team. In his current role, he provides training and architectural solutions for cloud-based environments, focusing on IBM Workload Deployer, WebSphere Application Server, and VMware-based technologies. Previously, he designed migration solutions for WebSphere Application Server and WebSphere Application Server Community Edition.

Rate this article

Comments

Back to top

Table of contents

• Customizing virtual application patterns
• Makeup of a plug-in
• Creating a custom plug-in
• Installing and using the plug-in
• In conclusion
• Resources
• About the authors
• Comments

Next steps from IBM

Learn more, join the cloud community

---

• Community: Cloud computing conversations led by IBM community members
• Buy: Try IBM software on IBM SmartCloud Enterprise

Dig deeper into Cloud computing on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers