Before you start
In many companies today, the development environment is split between distributed and host-based development. Over the years, there have been efforts to provide some common tooling but, in general, the divide has remained. With the ever-increasing relationship between parts on different systems, this divide is becoming more of a problem. With the latest release of IBM® Rational Team Concert™ in combination with Rational® Developer for System z® and other integrated development environments, we now have a common development environment to be used for all platforms, including IBM® z/OS®.
Developers have known for a while that Rational Team Concert can be used for distributed development, and there are many examples of the value that this provides. This article explores the new capabilities related to the z/OS platform.
You can use Rational Team Concert for change management, including defect tracking, planning, reporting, dashboards, for software configuration management (SCM), and for build management. Organizations can choose to use the capabilities that they need for any given project. Many organizations start with change management, planning, reporting, and dashboards. This provides the foundation for team collaboration, and transparency of the development process within an organization. These parts can be used in combination with any existing SCM and build infrastructures, and they provide the required functions for development for commonly used languages and platforms. The work item, planning, and dashboard functions already provide the required capabilities for the z/OS environment. SCM and build processes, however, require platform-specific capabilities. The SCM and build capabilities are where enhancements have been made to make Rational Team Concert a truly multiplatform tool. For more information about what this software offers, see the links in the Resources section.
Although this article focuses mostly on the SCM and build features and the additional capabilities specific to z/OS, we will start with the basics to correctly position the capability.
The Rational Team Concert server
The first concept to understand is the Rational Team Concert server. This server can be running on any platform supported by Rational Team Concert and backed with any supported database. The platform choice for the server has no effect on which platforms you use for application development.
For each machine or platform that you plan to use for a build, you must install the Rational Team Concert build components on at least one system where builds for that type of system take place. Standard combinations include the Rational Team Concert (RTC in the diagram) server on Linux for IBM® System z® and the database on DB2 for z/OS, as shown in Figure 1, or Rational Team Concert server on IBM® AIX® with the database on DB2 on AIX.
Figure 1. Rational Team Concert server diagram
After you have a running server, defining a Rational Team Concert project area is the next step. Project areas group all of the related artifacts together. Don’t be confused by the word project, because you do not have to create a new project for each release of your application. A project area is a way to group code, people, and tasks that will all be managed by using the same process.
A Rational Team Concert server can host multiple project areas with different processes. People can and often do participate in multiple project areas. For example, we develop Rational Team Concert by using Rational Team Concert, so we have one project area that is used for the development of Rational Team Concert for all releases, maintenance, and so forth.
When a project area is created, it is based on a defined process template. The template contains the process definition. The process template defines the types of work items, their states attributes, and actions that can occur as work is done on source code. Rational Team Concert includes two process templates; one called Scrum and based on scrum project management methods, and one called the Formal process template, based on traditional planning (waterfall) methods,. Start with the template closest to your current process or the one you want to use, and modify it to suit your needs. There is much detailed information about the process templates and modifying them in "Customizing the process in project, teams, and templates" in the "Working in the Eclipse client" section of the Introduction to Rational products information center (see Resources for a direct link). The Formal process template is what many traditional developers start with; however, if you are interested in moving toward agile or iterative development, you might want to start with the Scrum template.
In the project area, you assign which users who will be administrators of the project and members of the project, as well as assign them to their roles. Each role is granted a set of permissions in the project area. This allows you to assign permissions to users through roles, rather than individually. You can customize the project artifacts, such as new roles, different permissions for roles, new work item types, or different workflows in the project area itself, or you can modify the artifacts in the template for any project area that is already created.
After the project area is created, the next two things to think about are the source control and build. Let’s start with the basics, and then move into how they are used with z/OS artifacts in particular.
For source control, it is important to understand Rational Team Concert uses a stream, component, and change set method (terms definitions follow). The source files are stored in the repository (the database) as the definitive source and are extracted to a file system for editing or building. You can work locally with your files, and then have a build agent load them onto the appropriate system for building.
Components are the basic collection of files that Rational Team Concert deals with. Components are generally made up of projects, such as Eclipse or Microsoft Visual Studio projects, for easy interaction with the Rational Team Concert clients. Do not confuse this project with the Rational Team Concert project area nor with the general term project. This is an Eclipse project, just as a Java project, a zComponent project (specifically for files to load to z/OS), a C/C++ project, or an RPG (Report Program Generator) project can be types of projects for Eclipse-based environments.
Specifically for z/OS, the zComponent project (which is a kind of Eclipse project) is used to group files that are then grouped into Rational Team Concert components. The zComponent project has been created to allow the association of the metadata required for files to be loaded to z/OS.
Users load components or Eclipse projects on their clients, so it is important to keep the number of files in the projects and components to a reasonable size. Additional specific recommendations are provided in the migration section that follows.
Figure 2 shows a sample zComponent project with representative folders and files. zComponent project is also referred to as a zProject as in this screen segment.
Figure 2. Sample zComponent project
Streams are created from components. For each stream, you select which components, at which level, should populate the stream. Streams contain pointers and information about the components that they include. It is important to understand that streams do not include full copies of the files; therefore, multiple streams do not mean additional disk space used on a system.
You can use steams to group components at a particular level. For example, you can have a production stream that contains the components at the level that is currently in production (all of the files within a component have their own revision levels). The stream tracks the revision level of each file in each component that it contains. You can then have a QA (quality assurance) stream and a Dev (development) stream, each with the same components, but each at its own level.
In this example, all streams contain all of the same files, but each at its own revision level. Figure 3 is a flow diagram that shows such a stream structure, where changes flow from Dev to QA (test) to Prod (production) streams.
Figure 3. Stream flow diagram
One other important concept is the repository workspace. It s defined to interact with streams and components and to help developers manage changes to code. It is useful to consider repository workspaces as the server copy of the user's sandbox. A sandbox is where files are loaded on a file system for a user to directly interact with them for editing or for building through a build engine.
By using the repository workspace, users can easily move between machines or clients yet still access their most recent changes. Generally, users have their own repository workspaces, and repository workspaces are created for each build definition, as described in the Build section.
Because repository workspaces are owned by a person, and they can be set to Private (only the individual who owns it can see the contents), Team Access (members of specified team can see the contents), or Public (anyone with access to the project area can see the contents).
These workspaces are used to save code changes into change sets for building and testing before they are delivered to the streams. Change sets are a grouping of changes that can be delivered together. This provides for a good way to save changes that can then be shared with others for early development or for code reviews.
The next major area is the build. Rational Team Concert provides many capabilities for calling existing build tools, as well as providing a feature-rich, dependency-based build. The dependency-based build is based on a set of language scanners that can process the code in the SCM to determine compile-time dependencies. For example, it will find all of the COBOL programs affected by a copybook. For languages to build on z/OS (such as COBOL, PL/I, and Assembler), it is best to use the dependency-based build capability. For other languages, you can use Apache Ant or Make build options.
Rational Team Concert also provides a build definition to submit job control language (JCL) on z/OS and an Ant Enterprise Extensions build definition for running Ant builds on z/OS. These are beyond the scope of this article, but see "Building with Jazz Team Build" in the Rational Team Concert documentation for information about those (see the Resources section for a link).
To use a dependency-based build, there are a few more concepts to cover:
- Language definitions
- Data set definitions
These are defined by an administrator or build administrator. These definitions are shared across all project areas on a single Rational Team Concert server, so it is better to create them in a single project area to simplify maintenance.
A language definition is a specification of a set of steps (defined as translators) to compile source files of the specified type, and the scanner to be used for those files to understand the dependencies.
Rational Team Concert provides a set of default scanners, for COBOL, RPG, and Assembler to name a few.
Figure 4. Sample language definition
The translator provides the details necessary to perform a step such as a compilation for a build. This includes specifying the actual compiler used, the syslib specification, and all required input and output data sets.
Figure 5. Sample translator
Data set definitions
These provide the specification for the data sets used as input and output data sets, as well as the actual locations for compilers or other programs that will be run by the build. The screen captures in Figures 6 and 7 show examples.
Figure 6. Sample data set definition for loading SCM members
Figure 7. Sample data set definition for specifying a compiler
With these definitions in place, you can create a dependency build definition. The build definition specifies the repository workspace to use for the build. This repository workspace is associated with the stream that it will be building, which language definitions to process and their orders, and the build engine that will be used to run the build. The language definitions that you include indicate which file types from the zComponent project will be built. The build engine definition stored on the Rational Team Concert server points to a physical system or LPAR (logical partition) that has the build toolkit installed and the Rational Build Agent running. Figures 8 and 9 show examples.
Figure 8. Sample build engine
Figure 9. Sample build definition
With all the build definition in place, users can submit builds. When a build is submitted, the system uses the dependency information described hereafter to calculate which files should be built. The first time that the build is submitted, it will build all of the file types specified by the build definition. This first build populates the build metadata that is used for subsequent builds.
There are just a few other important concepts to understand related to a build.
Source code data
The first is source code data. You begin by setting up the source code data for scanning. Within the Rational Team Concert Team Artifacts view, there is a section called Enterprise Extensions. Within that, there is the source code Data section.
Right-click to select the streams for scanning. Select all of the streams that will be built by using a dependency-based build. After you have selected the streams for scanning, the background scan starts. The background scanning takes place based on the schedule determined in the server administration view. This scanning produces the dependency information that is used for a dependency-based build. This dependency information can also be useful for developers when they are making program changes, so they understand what other files might be affected. This is why the software provides a query capability.
In the source code data, you will also notice that you can create a new query to access the dependency information that has been stored. The default scanners will capture information about the language, type of file (main program or include file), and the number of lines in the file, as well as dependencies. You can create queries to find certain types of files or to search for files by specific names. When you create a query, you can save it and share it with others.
Users can also add their own information to the source code data, and organizations can build their own scanners to add additional dependencies or other information. The query function can then be used to search this additional metadata, as well.
One way that queries are used is to find the location of a file to be changed. If developers are unfamiliar with the code, they can use the queries to find files based on naming conventions. By using this search, developers will know which component and which project in that component includes the files that they want to change.
Figure 10. Example of a source code data query, with results
Promotion is used to move source code and the related built artifacts up a promotion hierarchy. The stream flow diagram in Figure 3, represents one such promotion hierarchy that is defined. Promotion is used when both source code and build artifacts need to be promoted. With promotion, after the modules are built, they can be deployed and flow through the hierarchy without being rebuilt at a higher level. The promotion definition also supports source-only promotion for use at levels of the hierarchy where builds will be redone. If build output will never be promoted, the standard stream-to-stream delivery of source code changes can be used and would be preferable.
Promotion definitions should be created to move between levels of the hierarchy, both up and down. For each promotion definition, a set of build definitions needs to be specified. Build definitions need to be created at each level of the hierarchy to provide the high-level qualifier and the target stream that you are promoting to. This specification of a build definition might never need to be used to for a build, but the definition is used so that promotion functions can move files to the correct locations.
Figure 11. Sample promotion definition
Starting with Rational Team Concert Version 3.0.1, work item-based promotion is available. Users can select the work items related to the functions that they want to promote, and the software calculates the applicable change sets, based on the work items selected. From these change sets, the promotion function calculates the build output to promote. (The change sets indicate which build they were included in, and the result for that build will indicate the build output created for the files included in the change sets. Promotion will deliver the change sets to the next stream, copy the build maps for each built file, and copy the actual built output on the z/OS system to the new data set location.)
When promoting a set of changes using work item-based promotion, if the promotion will cause an out-of-sync condition, such as promoting only part of a change as determined by the build, the promotion will indicate missing work items that need to be included. When the Ignore Changes flag is set on a file, it can affect the way that promotion determines whether an out-of-sync condition will exist. For example, if the Ignore Changes flag is set on a copybook that is in the stream that you are promoting to, which is at a higher level than the one used to build the program, the promotion can be set to allow that promotion to happen. However, if the copybook does not have the Ignore Changes flag, the file will need to be rebuilt with the new copybook level before it can be promoted.
You can also promote entire components.
Component-based promotion is more often used to populate a new stream, whereas work item-based promotion is used to move individual changes.
Deployment is the last major area of z/OS-related function added. This capability is used to create a package from build results. The package is a set of files, data sets, or libraries to be deployed to a system or to a separate set of data sets or libraries. Deployment can be used to move packages between systems, or to copy files to a new location on the same system. Deployment is provided for z/OS, the IBM® i integrated operating environment, UNIX, and Linux platforms.
For deployment, an administrator creates package definitions and associated deployment definitions. Users can then create the packages as they make code changes, and deploy the packages to the various test environments, QA, and to production when they are ready. Deployment for z/OS provides both work item-based deployment and ship list-based deployment. Work item-based deployment allows users to select specific work items to have Rational Team Concert calculate the generated built output for the package. For ship list-based deployment, the user creates a file that defines the files to be packaged. Ship list-based deployment is available for all supported platforms.
The deployment package definition specifies the build definition where the changes have been built, as well as where the package is to be created, plus any data set name changes for when the package is restored (Restore Mapping file).
Figure 12. Sample package definition
For each package definition, you also create a deployment definition. This is what actually moves the package to the desired system and unloads it. You define multiple deployment definitions for each package definition to allow for the deployment of the package to the multiple test areas, as well as production. The deployment definition specifies what system the package will be deployed to through the build engine that you specify.
Figure 13. Sample deployment definition
Migration to Rational Team Concert
Now that you have a general understanding of the functions, it’s time to think about migrating to Rational Team Concert. Migration requires a conscious effort to move from traditional host SCM to a new way of working. This is a significant and exciting change and needs to be done in an organized way. Many organizations choose to begin with a coexistence solution to allow the developers to change only part of their processes first. With this approach, developers start with the planning and work item parts of Rational Team Concert while still using their existing configuration management system (SCM). Given that SCM migrations are not quick, this allows them to begin to see value from Rational Team Concert early. As the migration progresses, individual teams are moved into the SCM and build processes, thus taking full advantage of what Rational Team Concert has to offer.
When migrating from existing SCM software or library system, it is important to do the proper reorganization into components. To do this, it is important to understand the way that Rational Team Concert stores information in the SCM. Files are organized into Eclipse projects and then into components. When users load files to interact with them, they load by component or project. Therefore, it is important to organize related files together. For example, copybooks or include files that are related to only a small set of programs should be included with the programs in a project or component. However, copybooks that are used across an entire application should be put into their own component.
These concepts apply across multiple languages. With C/C++ code there are usually project-level header files in separate directories, as well as local header files that are used within several localized source files. Who is authorized to update files can also influence the component organization. If only a limited set of users is authorized to update a particular set of files, those files should be put into a separate component so that access can be set appropriately.
Determine the plan for setting up the component layout before any files are moved. It is best to try the organization in a test project area to make sure that it is done the way that makes the most sense for the end users who are interacting with the system. To facilitate the process of creating the zComponent projects and moving existing source code from a partitioned data set (PDS) into Rational Team Concert, the zImport capability is provided. zImport provides a customizable import definition to allow the import of files based on member and PDS naming conventions. Multiple types of files can be within a single zComponent project in separate folders to load them into separate PDSEs (partitioned data sets extended) on the host. Each folder specifies the low-level PDSE name. Multiple zComponent projects can load into the same PDS so that splitting files into zComponent projects does not have to represent or match the splitting of PDSEs. The user specifies the prefix when loading the files to z/OS during the load or inside of the build definition.
When considering the number of files to put within each of your components, keep in mind that someone using the Rational Team Concert ISPF client will load all of them, and many users are likely to load the entire component in their Rational Team Concert Eclipse clients, as well. Therefore, this should be a reasonable number of files for user interaction -- typically around 500 files or fewer. If it is a set of copybooks that are very rarely changed, then they can be in a large component, because it might not be loaded at all in a client.
When the files are loaded into Rational Team Concert, they should generally populate the production stream. Be sure to recompile the entire stream on migration, both to prove that it can be done with the build definitions and to prepopulate the metadata for a dependency-based build. After you have rebuilt the files, you can restore your existing build output if you choose. If you do, make sure that your build definitions have the Trust Outputs option selected.
After the production stream is populated, use the component promotion capability to populate the lower streams. This will copy the source, build artifacts, and the build metadata used by a dependency-based build for the lower streams. If you do not want to keep the output with each stream, you can clean up the outputs after promoting down to each level. If you do, you must ensure that your build definitions include the production libraries and, once again, specify the Trust Outputs option.
In loading your source code, you must also set up the correct data set definitions, language definitions, and translators. As part of the zImport process, you can associate these definitions with the correct parts, and then you can create the translators.
Much of what this article covered are administrative functions. Administrators need to learn the details of the Rational Team Concert items described here and create them for the users. Usually, users need to be able only to load workspaces, edit source files, run the required builds, test their code, and deliver the required changes. With Rational Team Concert role-based permissions, you can specify who can modify definitions and who can run them.
This article introduced the additional functions in Rational Team Concert for z/OS development. It also included several guidelines for how to use the capability and how to migrate to Rational Team Concert from an existing host SCM. See the citations in Resources for additional documentation and details about Rational Team Concert capabilities.
- Get details for Customizing the process in project, teams, and templates in the Working in the Eclipse client section of the Introduction to Rational products information center.
- See Building with Jazz Team Build in the Rational Team Concert documentation to find out more about build options.
- Learn more about Rational Team Concert:
- Find Rational Team Concert articles and links to many other resources on IBM developerWorks, and check the product overview page, features and benefits, system requirements, and the user information center.
- Check the Rational Team Concert page on Jazz.net.
- Watch the Using Rational Team Concert in a globally distributed team webcast or a demonstration of the Dashboards and reports, or listen to the podcast about IBM Rational Team Concert and Jazz.
- Visit the Rational software area on developerWorks for technical resources and best practices for Rational Software Delivery Platform products.
- Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics.
- Attend a free developerWorks Live! briefing to get up-to-speed quickly on IBM products and tools, as well as IT industry trends.
- Watch developerWorks on-demand demos, ranging from product installation and setup demos for beginners to advanced functionality for experienced developers.
- Improve your skills. Check the Rational training and certification catalog, which includes many types of courses on a wide range of topics. You can take some of them anywhere, any time, and many of the "Getting Started" ones are free.
Get products and technologies
- Download Rational Team Concert from Jazz.net and try it for free on up to 10 projects for as long as you want (requires registration).
- Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.
- Join the Rational Team Concert discussions or ask questions in the Jazz.net forums.
- Rate or review Rational software. It’s quick and easy.
- Share your knowledge and help others who use Rational software by writing a developerWorks article. Find out what makes a good developerWorks article and how to proceed.
- Follow Rational software on Facebook, Twitter (@ibmrational), and YouTube, and add your comments and requests.
- Ask and answer questions and increase your expertise when you get involved in the Rational forums, cafés, and wikis.
- Get social about thought leadership. Join the Rational community to share your Rational software expertise and get connected with your peers.