Speed up deployment of your industry solutions

Reduce complexity, cost, and manual steps

Industry solutions usually involve complex, distributed deployment with a manual approach that is often time-consuming and unreliable. In this article, learn how to reduce the complexity and cost of industry solution deployment through reuse and automation. The accelerated approach can benefit development and test teams, and can increase the consumability of the solution for customers. Learn the overall process from the perspective of project manager and architect. Best practices and an example scenario are included.

Share:

Zheng Wang (wangzzh@cn.ibm.com), Software Engineer, IBM

Photo of Zheng WangZheng Wang is a software engineer at the IBM China Development Lab. His current focus is on industry solution development, deployment, and SOA.



Zhao Zhuo, Software Engineer, IBM

Photo of Zhao ZhuoZhao Zhuo is a software engineer on the Industry Solution Development Team. She focuses on the design and development of integrated information core installation and deployment automation framework.



04 October 2011

Introduction

Typically, deploying a solution for an industry is complicated. It involves a distributed software and hardware infrastructure, and installation and configuration of various products and components. The knowledge and experience required to deploy industry solutions can be hard to find. It is very costly for all members of a solution development and testing team to develop all the skills required.

It is expensive and time-consuming to manually deploy and set up development and testing environments. The cost is directly related to the number of development and testing environments that the technical teams set up. Customers that adopt the industry solution tend to be dissatisfied if the deployment process takes too long, or involves too many resources.

In this article, learn an approach to accelerate industry solution deployment that will also reduce complexity and overall cost. To implement this approach, a dedicated solution deployment acceleration team is required. The solution deployment acceleration team should have the required knowledge and experience on industry solution deployment, and the skills to automate the deployment process. The team can produce an automated deployment solution that reduces the workload of the development team, testing team, and customers. A mature solution deployment acceleration team can ensure efficient deployment acceleration, with low cost, through accumulated reusable assets and experience. The team can effectively reduce the overall cost throughout the industry solution project life cycle.


Types of deployment

Industry solutions typically involve two types of deployment: base products and a solution build.

Base products
Products that serve as the middleware layer providing applications with a runtime environment and services, such as data store, high availability, scalability, messaging, ESB, security, transaction, and so on. The middleware layer has no relationship to the specific industry of the solution.

Base products can also include products that provide the base semantics, contents, and frameworks specific to the industry of the solution. Base products are not developed by the industry solution project.

Solution build
build developed by the industry solution project to solve specific problems in an industry. It includes applications, database definitions, libraries, and resource configurations that will be running on top of the base products' middleware and semantics layers to provide business services of the industry solution.

Deployment of both base products and solution build is typically a distributed deployment that involves one or more networks and multiple target machines.


Topology design

The topology design is the first input to the solution deployment acceleration team. Topology design is a type of architecture design for industry solutions. It includes topology models and complementary documents. Topology models define the detailed elements to be deployed for both the base products and the solution build, including: hardware, operating systems, middleware, configurations, and applications. Topology models also cover integration of all the components, including detailed parameters such as hostnames, database names, and JMS resource JNDI names.

Topology design is the backbone of the industry solution deployment. It is a communication platform upon which all the stakeholders form a consistent understanding of the exact result the deployment will produce. Industry solution development teams should collaborate with all the stakeholders, including the solution deployment acceleration team, to make the topology design as complete as possible before the team can start to implement automation.

IBM Rational Software Architect Version 8.0 provides built-in topology modeling functions that are powerful and easy to use. Figure 1 shows a simple model composed using Rational Software Architect.

Figure 1. Sample topology model
Sample topology model

(View a larger version of Figure 1.)

The model above defines the deployment topology of a simple Order Processing solution. It contains the following elements.

  • Three physical machines: DB2 machine, Dmgr machine, and Custom machine. Each machine runs a Windows® Server 2003 operating system.
  • On the DB2 machine, DB2 9.5 is installed. A DB2 instance is created on top of the product, and a DB2 database (OrderDB) is created in the instance.
  • On the Dmgr machine, IBM WebSphere Application Server 7.0 Network Deployment is installed with Fixpack 7.0.0.13. A deployment manager profile is hosted by the product.

    The deployment manager hosts a cell, cell01, in which a DB2 JDBC provider, DB2 JDBC data source, and J2C authentication alias are defined. The data source references the OrderDB on the DB2 Machine.

  • On the Custom machine, WebSphere Application Server 7.0 Network Deployment is installed with Fixpack 7.0.0.13. A custom profile is hosted by the product. The custom profile hosts a WebSphere node that is federated into cell01. The WebSphere node hosts an application server, server1, on which an enterprise application OrderEAR is deployed.

The model will help illustrate the process to create an automated installer based on the IBM Software Assembly Toolkit.

Typically, you can categorize topology model elements into different layers, as:

  • Nodes (physical or virtual machines)
  • Operating systems that can be installed on the nodes, such as Windows or AIX
  • Middleware and industry semantics products
  • Base product configurations, such as application servers, clusters, messaging, security, web proxies, industry content frameworks, and so forth
  • Databases, applications, and resources, such as JDBC and JMS resources

Rational Software Architect 8.0 does not provide all of the out-of-the-box model elements for the large number of products and configurations that may be used in industry solutions. It provides out-of-the-box model elements for basic IBM products and configurations, such as for WebSphere Application Server and DB2. Rational Software Architect provides excellent concepts and a framework for extending existing model elements and creating new model elements for deployment.


Documenting manual installation

The second input to the solution deployment acceleration team is the document outlining manual installation. The manual installation document is the first realization of the topology design. It describes, systematically and in technical detail, how to deploy the industry solution manually. For the topology design described above, the document should include the following.

  1. Prepare three x86 physical machines and install Windows Server 2003 operating system on each. Configure system time on the machines to be the same for creating the WebSphere cell.
  2. Set the three computer names in Windows as: db2Host, dmgrHost, and custHost. Restart the three machines.
  3. Prepare installation media:
    • DB2 9.5 on db2Host
    • WebSphere Application Server ND 7.0, WebSphere Application Server Fixpack 7.0.0.13, and WebSphere Update Installer 7.0.0.13 on dmgrHost and custHost
  4. On db2Host, install DB2 9.5, create a DB2 instance called DB2INST with DB2 User db2admin, and use the DB2 control center to create a database called OrderDB on the instance.

    Create a database schema that includes tables, views, procedures, and indexes in OrderDB using the control center.

  5. On dmgrHost, install WebSphere Application Server ND 7.0 and WebSphere Update Installer 7.0.0.13. Use WebSphere Update Installer to apply WebSphere Application Server Fixpack 7.0.0.13 to the WebSphere Application Server ND installation.

    Create a deployment manager profile on WebSphere Application Server ND with cell name cell01. Start the deployment manager and log on to the Administrative Console.

  6. From the Administrative Console, create a DB2 Universal JDBC provider in the cell scope. Create a J2C authentication alias orderDBJ2CAuth in the cell scope that uses user ID db2admin.

    Create a DB2 JDBC data source with:

    • JNDI name: jdbc/orderDS
    • Host name: db2Host
    • Database name: OrderDB
    • Database listener port: 50000
    • J2C authentication alias: orderDBJ2CAuth
  7. On custHost, install WebSphere Application Server ND 7.0 and WebSphere Update Installer 7.0.0.13. Use WebSphere Update Installer to apply WebSphere Application Server Fixpack 7.0.0.13 to the WebSphere Application Server ND installation. Create a custom profile and federate the WebSphere Node node1 into cell01.
  8. Log on to the Administrative Console. Create an application server called server1 under node1.

    Install the enterprise application OrderEAR onto server1, and start server1.

  9. Verify the Order Processing solution by opening a web page provided by the OrderEAR and creating an order on the web page.

To save space, the steps above are summaries of what would actually be in the document describing manual installation. The actual document should contain more details, steps, descriptions, and screenshots as needed.

The document covering manual installation is a validation of the topology design, and is a mandatory input for developing the accelerated deployment package described later. The solution deployment acceleration team should collaborate with the solution development team to test the manual installation document as much as possible before starting to implement automation.


Solution deployment acceleration team output: The deployment accelerating package

The deployment-accelerating package is the product, or deliverable, from the solution deployment acceleration team. A deployment accelerating package is the deployment automation solution for one topology design. There may be multiple topology designs for the industry solution, such as silver topology for a small business and golden topology for a high availability business. The project should have one document of manual installation for each topology design. In this article, we focus on how to create one deployment-accelerating package.

A deployment-accelerating package consists of, but is not limited to, one or more IBM Software Assembly Toolkit-based automated installers, installer-related supporting documents, and a deployment automation framework inside the solution build. The package contains all the information required to easily set up one topology of the industry solution. A deployer who uses the package doesn't even necessarily need knowledge of the industry or the industry solution.


IBM Software Assembly Toolkit-based automated installer

Why use IBM Software Assembly Toolkit? The toolkit-based automated installer is the core of the deployment-accelerating package. The installer that turns complex manual steps into automated click-and-go action, and hides complex technical details of deployment from deployers.

IBM Software Assembly Toolkit has one outstanding capability that makes it an ideal tool for automating deployments of industry solutions: distributed deployment. You can kick-off the installer on a staging server, and the installer is able to deploy various products and configurations to multiple target computers at one time.

IBM Software Assembly Toolkit has several advantages:

  • It provides rich methods for developing installer tasks. For example, you can use techniques such as IBM Installation Manager, Ant, Java™ programs, shell script, and so on.
  • Both the IBM Software Assembly Toolkit IDE used to develop installers and the resulting installers have friendly, easy-to-use GUIs.
  • It provides the base for reusable assets. The deployment community can build up a large reusable asset library. Installer developers can download and reuse existing assets from the library to speed new installer development.

To develop an IBM Software Assembly Toolkit based installer, you typically use the following steps.

  1. Plan the installer task outline.
  2. Reuse or develop application deployment accelerators for tasks.
  3. Create a solution deployment accelerator to assemble application deployment accelerators.

Plan the installer task outline

A task is a unit of deployment work that can be carried out on one target computer. The granularity and levels of task organization are usually different between manual installation and the toolkit-based installer. The toolkit-based installer uses two levels of organization for tasks: Task Group and Task. Figure 2 shows the installer task outline derived from the sample manual installation document. You can create the outline using the IBM Software Assembly Toolkit IDE, effectively creating a draft solution deployment accelerator.

Figure 2. Installer Task Outline
Installer Task Outline

Some actions in the document describing manual installation cannot be carried out by the toolkit-based installer, such as preparing three machines and installing operating systems, because the installer is running on an operating system. These actions will go into the installer supporting documents described later.

For each installer task, you need one or more application deployment accelerators to actually do the installation work.

Reuse or develop application deployment accelerators

As described in the IBM Software Assembly Toolkit Information Center (see Resources), an application deployment accelerator is a building block that you develop to automate a distinct unit of work, such as the installation of a product or a fix pack.

Application deployment accelerator is the smallest unit of an installer that can be assembled. You can have one ore more application deployment accelerators in one installer task, so the application deployment accelerators can collaborate to run on the same target computer to finish an installation task. Application deployment accelerators can be roughly categorized as follows.

Product installation deployment accelerator
For example, install WebSphere Application Server.
Configuration deployment accelerator
For example, create a JDBC data source in a WebSphere Application Server cell, or create and configure a DB2 database.
Utility deployment accelerator
For example, start WebSphere Application Server.

Best practice: Reuse as much as possible. Search the reusable asset library before writing an application deployment accelerator to perform a unit of automated work. If an existing application deployment accelerator matches your requirement, it will save you a lot of time.

Application deployment accelerators should be well designed to maximize reusability. For example, you can write one application deployment accelerator to install WebSphere Application Server at the same time as creating a WebSphere Application Server custom profile. However, this deployment accelerator is not reusable when people want to install WebSphere Application Server and create a deployment manager profile. It would yield better reusability if you write one application deployment accelerator to install WebSphere Application Server without creating any profile, and write one application deployment accelerator for each type of profile creation. You can then combine the application deployment accelerators to achieve various configurations.

To develop a new application deployment accelerator, you typically write silent installation, configuration, and utility scripts first. After testing the scripts, you wrap them using an application deployment accelerator so that the work can be assembled into the automated installer.

Best practice: Test the application deployment accelerator from the smallest scope. For example, if the main program of the application deployment accelerator is written in Java, test the Java program with fake input parameters. It's much quicker than writing a unit test solution deployment accelerator and using the deployment wizard to test.

The IBM Software Assembly Toolkit Information Center has detailed steps for developing and testing an application deployment accelerator. (See Resources.)

For the example Order Processing solution deployment, you'll reuse or develop the application deployment accelerators (projects whose icons are marked with A) shown in Figure 3.

Figure 3. Application deployment accelerators
Application deployment accelerators

Develop a solution deployment accelerator

As described in the toolkit's Information Center, a solution deployment accelerator organizes a collection of application deployment accelerators to provide a complete deployment solution.

A solution deployment accelerator organizes application deployment accelerators in three levels: task group, task, and application. A task group contains one ore more tasks. A task contains one ore more applications. Each application is a reference to an application deployment accelerator. You can pass parameter values to the application deployment accelerator through an application.

We already created a solution deployment accelerator that is used to plan the installer task outline (described in Plan the installer task outline). Now you incorporate application deployment accelerators into the solution deployment accelerator, as shown in Figure 4 and Figure 5.

Figure 4. Sample solution deployment accelerator – Part 1
Sample solution deployment accelerator – Part 1

(View a larger version of Figure 4.)

In Figure 4, you pass parameter values to application deployment accelerator (db2ese95_win) through variables defined on the application deployment accelerator.

Figure 5. Sample solution deployment accelerator – Part 2
Sample solution deployment accelerator – Part 2

In Figure 5, the Create JDBC Resources on the Dmgr Node task uses three application deployment accelerators.

Build and run the Installer

An IBM Software Assembly Toolkit based automated installer is built from a solution deployment accelerator and related application deployment accelerators. You can use the toolkit IDE to export the installer. See Resources for "Introduction to IBM Software Assembly Toolkit" in the toolkit's Information Center to learn more about the build process.

When you run the installer, you should see the Installer Deployment Wizard, as shown in Figure 6, Figure 7, and Figure 8.

Figure 6. Installer Deployment Wizard – Choose task groups
Installer Deployment Wizard – Choose task groups
Figure 7. Installer Deployment Wizard – Input parameter values
Installer Deployment Wizard – Input parameter values
Figure 8. Installer Deployment Wizard – Deploying
Installer Deployment Wizard – Deploying

The toolkit's Information Center has detailed steps for building and running an installer. (See Resources.)

Best practices for arranging installers

A few best practices to arrange your installers include:

Separate the base products installer from solution build installer.
A base products installer uses base products as the installation media, and a solution build installer uses solution build as the installation media. Logically, they serve in different parts of deployment.

When new versions of base products or new deployment topology are to be adopted, only the base products installer needs to be updated. A solution build installer has the opportunity to stay untouched. Likewise, when a new solution build comes out, the solution build installer might need updating, but the base products installer normally doesn't need to be changed.

If there are many deployment tasks for the industry solution, the separation becomes a mandate. A single large installer will be troublesome for development, testing, and maintenance collaboration between solution deployment acceleration team members. It will also appear too complex to installer users and create confusion.

Because it's very simple, the example solution deployment accelerator is not split.

Separate the solution build installer from solution build.
If the solution build is embedded inside the installer, every time a new build comes out the installer must be updated. Ideally, when a new solution build comes out, nothing needs to be changed in the installer. To help achieve this goal, you should maximize the usage of a deployment automation framework inside the solution build itself (described later).

Supporting documents for installer

Supporting documents for installers are also an important part of a deployment-accelerating package. Though installers are easy-to-use and highly automated deployment facilities, users of installers will also need supporting documents to:

  • Perform actions than cannot be automated.
  • Avoid confusion.
  • Get help when troubleshooting problems.

The installer should have the following supporting documents.

Installation overview
Describes the overview steps of the industry solution deployment based on the deployment-accelerating package. It also describes the result of the deployment, and might include illustrations of the simplified topology models.
Installer operation guide
Each installer in the deployment-accelerating package should have one Operation Guide. It should cover the prerequisites for running the installer. For example, users need to know about environment preparation, which includes hardware and operating systems preparation, hosts file (in Windows) editing, where to download what middleware medias, and so on. It should also include the meanings of input parameters to the installer.
Non-automated installation and configurations
Usually, the installations and configurations for all the products can be performed by the automated installer. However, there are occasionally some product installations or configurations that do not have a public API to be used by the installer.

At the time of this writing, you can incorporate a screen action record and replay tool, such as Rational Functional Tester, to implement this part of the automation. Customers typically don't want to purchase additional tools. You might not always be able to provide manual installation documents.

Installer manual
Describes actions and results of each task in the installer, and how to recover from failure of each task. It may also include information for troubleshooting general problems during deployment, such as a network connectivity problem between a deployment wizard and deployment agent of the installer.

Deployment automation framework inside the solution build

To help separate the installer from the solution build, you need to use a deployment automation framework inside the solution build. The framework includes parameter files, parameter file parsing, and execution logic. The invoker (either a manual deployer or the toolkit-based automated installer) uses the parameter files to tell the framework how to perform the solution build deployment on a single machine.

Below are two examples of how to use the framework.

  • To install applications: Use a property file as the parameter file, which contains properties, to specify the application EARs to be installed and how to install each of them. The automation framework (such as jython scripts) parses the property file to get all required information and use related EAR binaries in the solution build to install the applications. In this case, the parameters (properties) are static values.
  • To set up a database: Use a Windows bat file as the parameter file to set some variables, such as database user name and password. The automation framework, such as another bat file, will read variable values from the bat file (by invoking it), and run all the database scripts (sql, ddl files) in the solution build to set up the database. In this case, the parameters (variables) are dynamic values provided during deployment.

To implement an automation framework, it's preferable to heavily use scripts, such as Windows bat, python, ANT, sql, and so forth. The solution deployment acceleration team is responsible for developing such a framework, and for training the development and testing team to use the parameter files to specify behaviors of the framework for deploying the solution build. Eventually, the team will be able to accumulate the script code into a reusable script library. The cost of implementing the accelerated approach described in this article in new industry solution projects will be greatly reduced.

You cannot accomplish all the tasks of the solution build deployment with only the automation framework. You have to perform the deployment in a distributed environment. Additionally, some parts of the solution build deployment cannot be covered by the framework due to the limited power of scripting languages. You can use the toolkit-based automated installer to automate all of the solution build deployment. The installer will: collect dynamic parameter values from the deployer, distribute required files to the target machines, and kick off the framework on the target machines. The powerful installer will also run parts of the solution build deployment that are difficult to do with the framework.

The parameter files have to be put inside the solution build. The parameter file parsing and execution scripts of the framework can be put either inside the solution build or inside the toolkit-based automated installer. It is recommended that you also put it inside the solution build to realize the following benefits.

  • Help the development and testing teams deploy the solution build when a compatible installer is under development.
  • The framework can be used by automation mechanisms other than IBM Software Assembly Toolkit.
  • The framework code is open to every stakeholder, so the stakeholders can better understand the mechanism and contribute ideas to improve the framework, even without knowledge of the toolkit.

Deployment accelerating package build and release

A deployment-accelerating package build is a package that contains all the information needed for a tester to set up the industry solution. It contains toolkit-based automated installers, installer supporting documents, and the deployment automation framework. The package tester tests the deployment-accelerating package for defects and inconsistencies. The package tester is not supposed to have in-depth knowledge of the industry solution.

A tested deployment-accelerating package build is the interim work of the solution deployment acceleration team. Target users of the package build include industry solution developers and testers.

If a deployment-accelerating package build contains complete automation artifacts for the whole industry solution deployment, and is fully tested and verified, you can call it a deployment-accelerating package release. A release is consistent and complete, and contains all the correct information required to set up the industry solution in a highly automated fashion. The deployment-accelerating package release must be easy to use. People from other teams or industries need to successfully deploy the industry solution without in-depth knowledge of it.

A deployment-accelerating package release is the solution deployment acceleration team's deliverable. Target users of a deployment accelerating package release include industry solution onsite service teams and customers.


Life cycle of a deployment accelerating package

A deployment-accelerating package typically passes through several phases in its lifetime. The collaboration model used by the solution development team, testing team, and the solution deployment acceleration team should address the entire life cycle.

During the industry solution development, a new topology design, such as a golden topology, and corresponding new document for manual installation might emerge. Subsequently, the team might be developing multiple deployment-accelerating packages at the same time, with each package possibly in different phases. The project manager should plan for concurrent, multiple packages. The activities in each stage of the life cycle of one package are summarized below.

Creation
Is from zero to the first stable deployment-accelerating package build. Inputs include a basic topology design and a document for basic manual installation. The package creation phase may span several iterations in the project, depending on the size of the deployment and the resources of the solution deployment acceleration team.

The development and testing team rely on the manual installation document to deploy the industry solution for development and testing. Ideally, this phase is finished early so the package can serve the development and testing team early and maximize the benefits of automation being brought to the whole project.

The industry solution development team might get new requirements for the topology design, resulting in updating requirements for the manual installation steps and the deployment-accelerating package. New or updated requirements should be managed in a pool specific to this package. The solution development team prioritizes the updated requirements and puts them into this pool. The team will implement the updated requirements in the next phase.

Growing
A basic deployment-accelerating package is already created, and begins to serve the development and testing team.

As the solution progresses, new installation and configuration requirements arise. The industry solution development team puts the new or updated requirements into the pool and maintains their priorities. The solution deployment acceleration team selects items from the pool according to priority and implements them into the package.

The team may produce a new or updated package each iteration or every two iterations. The solution development and testing teams will use the newest version of the package, and the manual installation steps that have not been implemented into the package, to set up their development and testing environments.

Stable
The deployment topology design is stable, with all items in the pool implemented. There might still be potential update requirements. At this point, the project manager tends to reduce solution deployment acceleration team resources assigned to this deployment accelerating package.
Finalized
After the industry solution's final release, the deployment accelerating package is also finalized as a package release. The solution deployment acceleration team assigned to this package changes focus from developing the package to its release maintenance and customer support.

Deployment environment

For the solution deployment acceleration team to test deployment accelerating packages, and for the development and test teams to do their daily work, the project typically uses VM farms as running environments. A VM farm is a collection of VMWare images managed by a VMWare infrastructure.

Deployment of an industry solution may involve many machines. Using a VM farm can greatly reduce the cost of setting up the deployment environments (you don't have to buy a lot of physical machines). A VM farm also provides facilitating functions that can increase efficiency in managing a deployment environment. It provides:

  • Flexible images assignment and revocation functions to help dynamically arrange computing resources among development, the test team, and the solution deployment acceleration team. Managing VM images is much easier than using pure physical machines.
  • Snapshot functions to help maintain in-progress milestone environments for testing deployment-accelerating packages step by step.
  • Back-up and replication functions for successful industry solution deployments.
  • An automation framework for further improving the deployment process. For example, it can automate the VM images creation and automate sequential invocations of multiple installers in a deployment accelerating package.

Summary

In this article, you learned about an accelerated approach that can greatly reduce the cost of deploying an industry solution. A dedicated solution deployment accelerating team implements the approach. The team produces a deployment-accelerating package containing complete and easy-to-consume artifacts to help deploy the whole industry solution in a highly automated fashion.

The accelerated approach is suitable for the following situations.

  • Complex, distributed industry solution deployment that mainly depends on manual operations.
  • A department with long-term focus on similar types of projects (for example, WebSphere Application Server or WebSphere Process Server based industry solutions).
  • An industry solution as a product, where the same solution will be delivered to more than one customer. Reusability of the deployment accelerating packages is a big benefit. Solution migration will also realize large benefits from partial reuse of the packages.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational, WebSphere
ArticleID=762476
ArticleTitle=Speed up deployment of your industry solutions
publish-date=10042011