DevOps is a set of principles and practices designed to help development, test, and operations teams work together to deploy code more frequently and to ensure a more effective feedback loop. The practices include iterative development, deployment automation, test automation, release coordination, monitoring and optimization, and many more. This article describes the factors to consider as you build a deployment automation solution for an enterprise that has applications that run on multiple platforms, including the mainframe.
Manage the deployment of multi-platform applications
Although DevOps principles apply across all platforms, the shared nature of the IBM z/OS environment has shaped, and sometimes constrained, the deployment process. In the current z/OS environment, deployment is generally automated consistently across all environments. However, this capability cannot extend to other platforms because the tools are specific to the z/OS platform. In the z/OS environment, the tools that manage source code also provide the build and deployment capabilities. Because these tools have been in place for many years, they have been significantly customized.
In the current multi-platform environment, composite applications drive the need to coordinate the deployment of the entire application across various platforms. The deployment capability in place for the z/OS environment does not coordinate well with other environments. A comprehensive and automated deployment solution is not available.
At the heart of a multi-platform application deployment system, you might expect to see a consolidated inventory view, which shows you the application with all its components and subsystems, mapped to the deployment environment. The inventory lists where each logical component is and where it should be, as shown in Figure 1 in IBM® UrbanCode® Deploy.
Figure 1.A sample inventory view from UrbanCode Deploy
Manage the environment
A software project typically has a set of deployment environments such as development, quality assurance (QA), and production. An environment is a collection of resources as a deployment target, as shown in Figure 2. A resource can be a physical server, a logical partition (LPAR), a virtual machine, or a subset of a cloud. It can also be a logical deployment target, such as an IBM® CICS® region, a database, or an application server platform. The deployment system needs to understand and be able to model the environment before it can create and maintain an inventory of application versions mapped to environments. In the distributed platforms, most IT organizations use application-specific environments, but multi-tenant servers can be the targets of multiple applications. The mainframe environment is typically highly shared. Approvals and team processes are typically scoped to environments.
Figure 2. UrbanCode Deploy sample environment
Design the deployment processes
Deployment of applications generally consists of many steps across a large set of varying technologies. Some middleware products, such as IBM CICS and IBM® WebSphere® Application Server provide deployment tools of their own. However, these technologies are specific to the middleware and do not span multiple middleware environments. In the past, teams have developed scripts to deploy applications into production for the distributed environment. In many cases, these scripts pull the binaries from the build system to feed the deployment activities. In the z/OS environment, deployment has been handled by the source control systems. Most organizations have only a deployment checklist with many manual steps and with no automated coordination between platforms or environments.
"Automate everything" is one of the DevOps principles. For deployment the goal is to automate the process so that each deployment is done the same way. The process is coded and version controlled so that the deployment engineer or administrator defines it only one time. The process can be run whenever a deployment is required by anyone who is authorized to deploy into that environment. By having one way to deploy to all stages of development, any deployment-related problems are identified early in the development cycle.
One option is to add version control to existing scripts and require that they be used at each stage of development. Although this option is an improvement, scripts can be difficult to maintain. It is hard to clearly see the process, because it is hidden in the code.
The ability to define templates for each process and use those templates for any deployment of that type helps ensure consistency across the environments. This method is easier to maintain, long term. A graphical process designer can help to visualize the process and make it easier to understand and maintain.
Manage the versions
Frequent deployments require the deployment capability to manage versions across the system, including the deployed version of the software, the environment (sometimes software-defined), and the deployment process. To roll back to the previous stable version, you need the same environment and deployment process of that version, and you also need the software binary code.
For multi-platform applications, the deployment picture can be even more complex. Depending on the change, you might have to deploy all the parts together or you might have to roll back all the parts together. The deployment system needs to be able to tag or take a snapshot of a release version for multiple components and for all the artifacts involved in the deployment of a release.
Traceability is key for effective application deployment. Organizations need to be able to trace what is being deployed back to the original requirement for that change. When there is a problem in production or during test, traceability makes it easier to understand what changes are going in that could have caused the problem. The deployable artifact needs to be linked to the build that created it, the requirement that it was implemented for, and the tests that were run to validate the function.
Incremental versus full deployment
One approach to deployment is to deploy the entire component each time, for example the application WAR file or JAR file. The other approach is to deploy only the specific changes. In the z/OS environment, organizations typically build only the changes to the application and deploy them into the environment. It is common to find that some parts of an application have not been rebuilt in over 20 years. In distributed environments, typically the complete application is rebuilt and deployed. Some reasons for the difference include the differences in language and size of the applications, the capability of the available tools, and the risk factors associated with change. Teams are hesitant to change what is working.
These deployment methodologies are not always followed. Some organizations rebuild their entire z/OS applications on a periodic basis. Some organizations deploy only the changes to their distributed applications.
Both methodologies exist and must be supported by the deployment capability.
z/OS deployment scenarios with UrbanCode Deploy
UrbanCode Deploy, formerly called uDeploy®, was developed to address the challenges of modern application deployments. Users model the structure of a multi-tiered and multi-platform application and orchestrate the processes required to deploy it. Environment-specific configuration provides the ability to create approvals and consistently promote the application through multiple environments as the application is developed. An easy-to-use graphical interface aids users as they design processes (such as the installation, uninstallation, and rollback processes) using drag-and-drop integration steps. The interface makes it easier to run deployments and monitor environment inventories. UrbanCode Deploy V6.0.1 includes z/OS support. This article describes two UrbanCode Deploy z/OS deployment scenarios:
Figure 3 shows a fairly standard enterprise application topology. The z/OS components (dark blue) run on CICS with virtual storage access method (VSAM) for the data source. Some of the web and mobile components (light blue) also run on z/OS using an application platform such as IBM® WebSphere® for z/OS. Some components (yellow) run on distributed platforms.
Figure 3. Sample enterprise application topology
The deployment of this entire application can be modeled and managed with UrbanCode Deploy. The UrbanCode Deploy z/OS agent can run deployment activities directly, or interact with an existing deployment tool, usually the SCM/Build system, to deploy.
Deploy directly using the z/OS agent
The UrbanCode Deploy z/OS agent is a process running on z/OS, which can perform a wide range of deployment activities such as performing file operations, deploying to application servers, running shell scripts including REXX (and many more), and downloading binary from one of the many build systems that integrate with UrbanCode Deploy.
To start, define the application, the components, and the environment, and
create the mapping of application to environment. Figure 4 shows the JKE
sample application. A
DEV environment is defined for the
application. At this time the application contains a single component
JKE Mobile Component and it has already been deployed
Figure 4. Model the application component and the environment in UrbanCode Deploy
Figure 5 shows the
DEV environment, its resources, and how the
components are mapped in UrbanCode Deploy.
ZOS84 is a
deployment resource, which is a z/OS LPAR with the agent running on it.
JKE Mobile Comp
nent is added
to this resource. The mapping tells the system that when this component is
DEV, it is going to be deployed to
Figure 5. Environment view of the JKE sample application
Use the graphical process designer to create a deployment process for the component, as shown in Figure 6.
Figure 6. Sample process to deploy a web application
In this example, the process performs the following activities:
- Downloads the deployable binary, which is a web archive (WAR) file from the repository. The build system is already integrated with UrbanCode Deploy. A new build, when available, is pushed to the repository.
- Unzips the WAR file into a temporary directory.
- Updates the web service configuration. The web application being deployed calls a back-end web service provided by CICS. The endpoint of the service is written in a web services description file (.wsdl) in the archive. This step is to replace the endpoint with the actual value of the target environment.
- Repeats Step 3 to replace the endpoint of another web service.
- Zips the content back to a WAR file.
- Stops the running web application in the application server.
- Undeploys the old version of the application.
- Deploys the new version of the application.
- Starts the updated application.
Each of the blocks is a plug-in step, or a reusable deploy task that you can drag and drop to your process and customize for your use by setting the appropriate properties. Figure 7 shows the properties of the unzip step in this process. This step unzips the MortgageMobileWeb.war file into the web folder under the current working directory.
Figure 7. Properties of the unzip step
In addition to a set of core plug-ins for common deployment activities, UrbanCode Deploy comes with many integrations for source control, issue tracking, testing, static analysis, and so on. The UrbanCode Deploy plug-in system gives you the ability to add your own plug-ins.
Unless your process has platform-specific steps, such as a step to run a REXX script, the process is cross-platform. For example, you can run the process shown in Figure 7 on any platform. Different environments can be on different platforms if required. The development environment might be running on Linux® and the production environment might be on Linux for IBM® System z® or z/OS in the above example. This arrangement makes it easier to change a platform when needed.
The next step is to create an application deployment process as shown in Figure 8 to assemble the component processes. The process is quite simple because there is only one component to deploy.
Figure 8. Application deployment process
The next step is to start the process and watch the progress. The timeline as shown in Figure 9 gives a visualized way to see the time distribution of each step. You can download the output log and error log of each step by using the icons beside the step.
Figure 9. Process execution status
Integrate with the existing z/OS deployment process
Most of the z/OS source code control tools provide deployment and promotion capabilities. Many organizations running on mainframes have a deployment process based on such tools. For these teams, the use of UrbanCode Deploy to integrate their existing systems is a low-risk solution to connect the platform locked-in deployment process to the rest of the environment and benefit from at least the combined tracking and coordination.
IBM® Rational Team Concert™ Enterprise Extensions provides source control, build, promotion, packaging, and deployment. This article shows how to use Rational Team Concert Enterprise Extensions and UrbanCode Deploy to deploy a z/OS application, using the same CICS and data components in Figure 3 as an example.
For this example, the package definitions are defined in Rational Team Concert. These definitions create archives of the data sets to be deployed. Corresponding deployment definitions deploy the data sets and run JCLs and REXX to complete the deployment, as shown in Figure 10. These are the processes to be integrated to UrbanCode Deploy.
Figure 10. Rational Team Concert package and deployment definitions
The first step is to define the application model with two more components in the application, the CICS component and the data component, as shown in Figure 11.
Figure 11. Application model with two additional components
When you define a component, decide what artifacts to store in the UrbanCode Deploy repository. Typically, you store the deployable binary. However, Rational Team Concert already has a way to manage the deployable binary in packages and understands how to interact with native z/OS files. Therefore, you can let Rational Team Concert take care of the storage and transfer of the package. In the UrbanCode Deploy component, a small meta-data file is stored, which keeps track of the package information.
Figure 12 shows the deployment process of the CICS component. It is a simple process with two steps. Download the artifacts, the meta-data file that describes a package, and deploy using Rational Team Concert.
Figure 12. Process to deploy using Rational Team Concert
In the deploy using Rational Team Concert step, set properties to specify the Rational Team Concert deployment definition to run, the ID of the package to deploy, and the connection information to Rational Team Concert in the advanced properties, as shown in Figure 13.
Figure 13. Properties of the deployment definition
The deployment process of the data component is similar to the CICS component except that a different Rational Team Concert process needs to be called.
The next step is to update the application process to include deployment of the two new components, as shown in Figure 14.
Figure 14. Application process with three components
If you have unique deployment tools or processes to integrate, the UrbanCode Deploy plug-in architecture provides an open framework to write your own integration plug-in. After you have managed the deployment in UrbanCode Deploy, you can take advantage of its rich features including inventory view, snapshot, graphical process editor, versioning, environment management, approval controls, and integration with a wide range of software development lifecycle (SDLC) tools.
In addition to the actual deployment, UrbanCode Deploy can integrate with other tools within the software development lifecycle. It integrates test automation into the deployment process to verify the deployment. Figure 15 shows an enhanced deployment process with test automations.
Figure 15. Application process enhanced to include tests
The test automation process uses integrations with IBM® Rational® Quality Manager, the test management tool, and IBM® Rational Test Workbench, the test execution and service virtualization tool, as show in Figure 16.
Figure 16. Workflow of test process
This enhanced deployment and testing process and the related products is shown in Figure 17.
Figure 17. Architecture diagram of deployment and testing products
ALT=Diagram of product relationships to application
This article describes the areas to consider for a z/OS and multi-platform deployment system. It includes two scenarios of how to use UrbanCode Deploy to help deploy z/OS applications. The first scenario shows how to deploy directly using the z/OS agent. The second scenario shows how to integrate with the existing deployment process. For more information about continuous integration, deployment and testing in z/OS, read the continuous integration for System z series of articles.
- More about Rational Team Concert:
- Find Rational Team Concert articles and links to many other resources on IBM developerWorks, and check the product overview page, features and benefits, system requirements, and the user information center.
- Check the Rational Team Concert page on Jazz.net.
- Watch the Using Rational Team Concert in a globally distributed team webcast or a demonstration of the Dashboards and reports, or listen to the podcast about IBM Rational Team Concert and Jazz.
- Find out more about Rational Test Workbench.
- Learn about Rational Test Virtualization Server.
- Explore more about Rational Development and Test Environment for System z.
- Explore the Rational software area on developerWorks for technical resources, best practices, and information about Rational collaborative and integrated solutions for software and systems delivery.
- Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics.
- Improve your skills. Check the Rational training and certification catalog, which includes many types of courses on a wide range of topics. You can take some of them anywhere, anytime, and many of the Getting Started ones are free.
Get products and technologies
- Download Rational Team Concert from Jazz.net and try it free on up to 10 developers for as long as you want (requires registration). If you'd prefer, you can try it in the sandbox instead, without installing it on your own system.
- Download Rational Test Workbench, which includes Rational Integration Tester, from Passport Advantage.
- Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment.
- Try building and deploying your next project on the IBM Bluemix cloud platform, where you can take advantage of pre-built services, runtimes, frameworks, application lifecycle management, and continuous integration.
- Get connected with your peers and keep up on the latest information in the Rational community.
- Rate or review Rational software. It's quick and easy.
- Share your knowledge and help others who use Rational software by writing a developerWorks article. Find out what makes a good developerWorks article and how to proceed.
- Follow Rational software on Facebook, Twitter(@ibmrational), and YouTube, and add your comments and requests.
- Ask and answer questions and increase your expertise when you get involved in the Rational forums, cafés, and wikis.