Make continuous deployment practical and cost-effective with Rational ALM tools

Use Rational software with the cloud to save time and effort, reduce risk, and improve governance

Continuous deployment means deploying software changes to a development, test, preproduction, or production environment. It is similar to adopting continuous compilation and then continuous integration to build every change. Steve Arnold outlines the three main challenges of this approach: design, automation, and governance. Then he explains how using specific Rational tools for ALM, in combination with the cloud, make it practical and cost-effective, because it reduces the effort of testing and deployment, improves governance, and lowers risks of deploying to a production environment.

Steve Arnold (steve.arnold@uk.ibm.com), Senior Technical Consultant, IBM

photo of Steve ArnoldSteve lives in Twickenham, London, with his wife and young daughter. He has worked as a technical consultant for IBM Rational since 2000, where he is a certified scrum master specializing in agile project delivery, modeling, and patterns-based engineering. He has written several articles covering the latest features in Rational Software Architect 7.5 and 8, as well as articles discussing lean development and various extensions to Rational Software Architect, Rational RequisitePro, and Rational Team Concert. Outside of work, he enjoys spending time with his family and studying, teaching, and practicing Tai Chi.



21 February 2012

Also available in Chinese

So what is continuous deployment, why is it interesting, and what problems does it help solve?

For many organizations, there is a dichotomy between the desire of the business and development teams to produce working software faster and more often, and the operations team's responsibility to manage the organization's infrastructure in a risk-averse way to ensure business continuity. Unfortunately, this dichotomy often results in a significant delay from when a development team "finishes" a piece of functionality to when it can be deployed to production. However, the delay is often necessary to ensure that the change can be deployed safely and will not result in loss of availability of that system. This delay is due to necessary rigor and the process required to reduce risks, because the deployment process often comprises many manual steps that have been done very few times before. Therefore, for many organizations, deployments are often done during weekends, are perceived as very risky, and often fail.

Continuous deployment is the process of deploying every software change to an environment, whether it is in development, test, preproduction, or production phase. It is similar to the way that many people use continuous integration to build every change. This approach means that whenever a change is checked in, the continuous deployment process also ensures that the change can be successfully deployed. So not only are the manual, risky steps removed from the deployment phase, but when you begin deploying to production, it is actually a common, everyday thing that is no riskier than compiling a piece of source code, rather than a new thing being done for the first time.

This provides many benefits to an organization:

  • There is less downtime and more upgrades, which results in better productivity and increased value from the software.
  • The business gets access to software changes sooner, so you improve the return on your investment in software development.
  • Testing teams spend less time configuring environments and deploying software.
  • Regression defects are found sooner, so they take less time and cost less money to fix.

In case this is sounding a bit far-fetched, another way of thinking about continuous deployment is that it is a logical progression of existing approaches. In the early 2000s, a couple of practices became common: continuous compilation, as produced by most IDEs (the code is compiled when the file is saved), and continuous integration (the code is built, and a set of unit tests are run when it is checked into source control). Each of these practices improves productivity. Just as continuous compilation improves individual productivity, and continuous integration improves a development team's productivity, continuous deployment is the natural progression that improves an organization's productivity by automating and reducing risks of releases. Small changes can be easily deployed, and you can get value from software sooner and more often.

Figure 1. Value of continuous compilation, integration, and deployment
Diagram compares the 3 types of continuous updates

Continuous deployment can provide many powerful benefits, but successfully implementing it has some unique challenges. The rest of this article looks at these in more depth, and explains how the IBM Rational ALM solution overcomes the difficulties to make continuous deployment practical.

Challenges in setting up continuous deployment

There are several challenges associated with creating a continuous deployment environment.

Designing and communicating the deployment

Communicating and designing the deployment often requires people from various disciplines (software architects, deployment engineers, operations, database administrators, project managers, and so forth), as well as from different departments and often from various locations, to communicate. The collaboration required is complex and needs input from many people. Often, important information is lost or never recorded, which can lead to failed deployments. Also, the lack of a consistent and complete way of describing the deployment inhibits people's ability to visualize and reason about the deployment, thus reducing the likelihood that nonfunctional problems (such as performance, backup, and scalability) will be spotted early.

In a continuous deployment approach, this risk is somewhat reduced by the intention to deploy regularly. However, in this situation, another risk emerges. What if your continuous deployment environment is different from production and you cannot reuse the script? (A very likely scenario.) Then, although you get many benefits through development, your production script could easily have defects, especially if the deployment is described in an ad hoc fashion. So there are two significant challenges:

  • How to define the key elements of a deployment in a way that it can be used to describe how the application is deployed to many different environments
  • How to efficiently get all of the interested parties to collaborate on the deployment plan to maximize the quality and reduce the risk and time involved

Automating the deployment process

Deployment today is significantly more complicated than it used to be, and most applications require configuration of middleware (creating data sources on an application server or setting up a message queue). This kind of automated configuration is potentially complex. It often requires specialized skills, and it is one of the reasons why many organizations have minimal automation to deploy applications.

After all, as the saying goes, if it were easy, everyone would do it.

Governing deployable artifacts

Finally, a common challenge is the governance of the artifacts to be deployed. There are two obvious approaches, but they both have flaws:

  • First, you can put each release into a designated place in the file system. This is pretty simple; however, this method can be fairly insecure, and it does little to ensure that the files you tested are the same ones that you deploy to production (they could easily have been modified or overwritten). So if you need to prove that what you tested is what was deployed, this is unsatisfactory.
  • The second option is to use your source control system to manage the files. This can at least provide the governance that you need. However, it also requires you to give the deployment team access to every project in every source control repository, and this aspect can become unmanageable in even a medium-sized organization.

Neither approach can help answer questions about the dependencies of applications to each other, to other components, and to open source libraries.

Although these challenges are significant, not setting up continuous deployment provides even more significant challenges:

  • It takes longer to deploy software and, therefore, to get value from it.
  • The business incurs increased risk when new software is deployed, and this can decrease productivity and sales, and even be detrimental to your reputation and brand, as well as to customer loyalty.
  • Costs can increase if you fail to meet nonfunctional requirements

How it works with the IBM Rational ALM solution

This section describes how three tools from IBM Rational software combine to satisfy the challenges described in the previous section.

IBM® Rational® Software Architect
A modeling tool that can be used to describe and collaborate on deployment topologies, as well as to generate deployment scripts
 
IBM® Rational® Automation Framework
A deployment automation engine that also provides libraries for common automation tasks
 
IBM® Rational® Asset Manager
A governed repository for deployable artifacts and standard infrastructure topologies
 
IBM® Rational® Quality Manager
A web-based test management environment for collaborative test planning, workflow control, tracking, and metrics reporting.
 

Designing and implementing the automated deployment

The first challenge is to build the automated deployment scripts that can be used throughout development and test, and also into production. This presents two challenges:

  • The first one is to communicate how the application should be deployed and how the middleware should be configured.
  • The second challenge is how to have a script or set of scripts support all environments, while still having confidence that all the scripts will deploy the application successfully.

Rational Software Architect provides the capability to describe deployment topologies in graphical manner. It also helps to verify that the described topology is consistent, makes sense, and is not missing anything that is required, thus it help reduce costly mistakes later on. After they are built, developers can collaborate on these topologies on the web, share them with stakeholders, and allow people to comment on them and participate in formal reviews.

Rational Software Architect helps solves the second challenge by providing the capability to define a logical and physical deployment topologies. This allows a logical topology to describe the important aspects of how the application should be deployed onto the various logical environments, and then it enforces any constraints in the logical model when it is mapped to real infrastructure.

Figure 2. Example of logical Java Enterprise Edition (JEE) deployment topology
Topology with components hosted on logical servers

This approach reduces deployment risks to production, because the model is used to generate the script for all environments, and it will be tested very thoroughly throughout development.

Even though scripts for different environments might vary, Rational Software Architect helps make sure that the scripts will correctly deploy the application, because the scripts are generated from the physical topologies, and it ensures that the physical topologies satisfy all of the requirements of the logical topology. So a script for a new environment can be easily generated, with a large degree of confidence that it will work first time.

Figure 3. Logical topology to infrastructure
Mapping of logical topology to physical machines

After you have defined how the application is deployed onto the middleware, it is possible to analyze the model for possible automation, matching the elements in the topology against known scripts to achieve that part of the deployment. This supports deploying to and configuring most IBM middleware and most common alternatives. This approach significantly reduces the effort of setting up a new automated deployment, plus it reduces risks of deployment to different environments.

The topology becomes the key artifact. Deploying to a different environment is a simple matter of mapping how the various application components (EAR, WAR, and DLL files, queue definitions, and so on) should be deployed to the new environment, and then generating the script to automate the work. So this requires much less manual effort, and the tools reduce the risk by verifying whether you have mapped the application correctly.

In the context of a continuous deployment, you will have one logical topology to describe the key elements of the application, and then one topology for each significantly different environment, which describes how the application should be deployed to that environment. Each of those topologies will be used to generate a deployment script for use in the continuous deployment process.

Delivering a change

After you have generated a deployment script, it needs to be incorporated into the development environment so that all changes are continuously deployed. This means that the test team saves lots of time, because they do not need to set up test environments (time for that is often estimated at 20-30% of the test effort). Even so, the deployment scripts get used and debugged, and the application is always potentially deployable to production.

The easiest way to set this up is to modify the existing continuous build script to store the deployable artifacts to an asset repository, and then to request a deployment with the assets from the repository. The deployment request will use the previously generated script from Rational Software Architect and deploy the application to a test target (or multiple targets).

Figure 4. Continuous integration engine requesting a continuous deployment
Publishing assets to RAM and starting deployment

You then configure IBM® Rational® Quality Manager to monitor the continuous deployment builds. On a successful deployment, it will then run your regression test software and unblock any tests that had previously failed for which fixes were delivered in the latest build. This approach means that every time a developer makes a change, the continuous deployment also verifies that the application is deployable and that all the regression tests pass. So in this environment, a simple one-line change could make the build ready to deploy to production in minutes.

Figure 5. Rational Quality Manager monitoring deployment builds
Flow diagram: monitoring deployment, running tests

One of the questions or concerns that people often come up with when thinking about this idea is the amount of hardware that they might need. However, there is a simple, cost effective solution: the cloud (see Resources for a link to the IBM SmartCloud Enterprise option). The deployment script starts an image, deploys the application to it, runs the functional tests, and shuts down the image. With this approach, organizations can cope with development periods when there are of lots of changes (the end of an iteration, for example), yet minimize costs when fewer changes are being made and less hardware is required.

Teams also need an environment for regular exploratory testing and building new automated test scripts. This can be achieved by setting up rolling environments, where a daily build is deployed and you keep several days worth. Or you can allow the test team to request builds on demand and then to deploy either to test hardware booked through the test lab management features of Rational Quality Manager or to a cloud, in the same way that the continuous deployment works. This approach can reduce hardware costs and provide development time savings due to reduced test setup, faster feedback, and lower risks of production deployments.

Figure 6. Rational Automation Framework deployment to multiple environments
Deployment to physical, virtual, or cloud hardware

Governing builds and promoting through environments to production

For a lot of organizations, the governance of the deployable assets (who changed what, when, and why, and whether it is an approved change) is often critical. That is either because of the importance of those deployed applications to their business or because of the need to demonstrate compliance with other organizations' mandates.

So how can IBM help with the governance of the deployable artifacts?

ITIL suggests the use of a "definitive media library" to manage deployable artifacts throughout the development process and to make them available to deploy to production. In the Rational ALM approach, we advocate storing the deployable files as assets in Rational Asset Manager and using the asset lifecycle and reviews for governance. This provides a centralized repository of all artifacts that have been released to production, as well as all artifacts that could be released. It also means that all deployment processes are driven from the same centralized repository, so they are the same except that the assets deployed to production have been through more reviews and approvals. And it means that the assets are securely held, so they can reach a deployable state only by moving through the various approval stages that are -normally related to testing stages:

  1. Identified
  2. In development
  3. Passed unit tests
  4. Passed system tests
  5. Passes user acceptance tests
  6. Staged
  7. In production
Figure 7. Rational Asset Manager asset review lifecycle
Asset states, from identified to in production

Another capability that Rational Asset Manager helps with is the storage of standard deployment topologies for use by development teams. Using this capability, the operations team can define "standard" configurations for development teams. This results in deployments that conform to a standard middleware configuration. The consistency makes deployments simpler and less costly to manage through the entire software development lifecycle.

The screen captures in Figure 8 show how an application architect searches for approved topologies in Rational Asset Manager from Rational Software Architect

Figure 8. Searches for approved topologies
Search and topology after the asset is imported

Summary

This article described the three main challenges of setting up continuous deployment: design, automation, and governance. Then it showed how you can use the IBM Rational ALM software with the cloud:

  • To increase collaboration on deployment design, which improves the quality and communication between the varied stakeholders
  • To use the deployment designs to automatically create deployment scripts for development, test, and production
  • To extend standard continuous integration practices to provide a governed, mature, continuous deployment practice that reduces the cost and risk of making a change to production, as well as to improve the speed of deploying changes

Hopefully, this article also manages to convey how the use of the Rational tools, in combination with the cloud, make this approach both practical and cost-effective to set up. Then you can quickly take advantage of the technique to reduce testing and deployment effort, improve governance, and to lower risks of deploying to production.

Resources

Learn

Get products and technologies

  • Download a free trial version of Rational software.
  • Evaluate other IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational, DevOps, Cloud computing
ArticleID=794772
ArticleTitle=Make continuous deployment practical and cost-effective with Rational ALM tools
publish-date=02212012