Build maps

Introduced in version 2.0.2, a build map is a new metadata store object that stores information about how a program was built, including Git repository information (URL, branch, snapshot hash), as well as additional source files and binary inputs from previous builds. It also provides information about the output binary files that are generated when the program is built.

Build map usage scenarios

In most cases build maps are not necessary for DBB to perform its core functions so creating them is optional. However, in some cases like the following scenarios, build maps can be useful.

Static link impact analysis gap

DBB impact analysis identifies which programs have a build dependency on source files that have changed and are to be rebuilt. Impact analysis also identifies main programs that produce statically linked load modules that have been impacted by included subprograms that have been rebuilt. Both of these types of impact analysis use the changed source file names to query the DBB metadata database to identify impacted programs. An impact analysis gap can occur when a load module contains statically linked objects whose names do not match the source file name that created them.

Beginning in version 2.0.2, DBB impact analysis incorporates build maps (when available) when performing impact queries on the output binary files of a program. These queries use the binary names that are provided by the build map instead of the source file name, thus closing the gap.

Processing a source file deletion

Prior to version 2.0.2, DBB build framework implementations can only make a good guess on what has been produced for a deleted program source file. Build maps allow for an accurate query of what has been produced for a given artifact and allow users to implement a "decommissioning" process including the removal of all produced outputs.

Deployment manifests

Build maps can also be used to create detailed application and package manifest files that are needed by deployment tools such as IBM Wazi Deploy and IBM DevOps Deploy.

Build audit requirements

Many z/OS customers have software build audit requirements that include being able to identify all the source files that are used in creating a given load module. This also includes providing SCM information like repository snapshots, branches and source file versions. Build maps can provide all this information.

Creating a build map

As build maps are database artifacts, an active metadata store connection is required to create and modify them. For more information about initializing the MetadataStore utility class, see Metadata store.

Build maps belong to a build group that must have already been created:

BuildGroup buildGroup = MetadataStore.createBuildGroup("MortgageApplication");
BuildMap buildMap = buildGroup.createBuildMap("MortgageApplication/cobol/epscmort.cbl");

Note: Build maps are immediately added to the metadata store when they are created. As creating a build map for a program replaces the existing build map in that build group, you must make sure the program successfully built before creating the build map of the program.

Populating build map fields

The build map can be used to store several types of build metadata data that are produced or collected while a single program is built.

Basic - Singleton properties that identify the build map program, group, build result and description
Sources - List of source repositories used during the build including branch and snapshot information
Inputs - List of resolved and unresolved inputs from dependency search including SCM information
Binary Inputs - List of resolved and unresolved statically linked compile units from the load module scanner
Outputs - List of outputs gathered from DBB execute DDStatements. Member and dataset fields are required for bridging impact analysis gap.

Note: All the build map fields are optional except the buildFile and group which, when combined together, create the unique database key for the build map. Additionally, in order for the DBB impact analysis to address the impact analysis gap mentioned above, there must be an outputs list with dataset and member names listed to search.

Basic

buildFile:   MortgageApplication/cobol/epscmort.cbl
group:       MortgageApplication-main
result:      build.20231026.041748.017
description: Build 37

In addition to the required buildFile field, you can also provide the name of the build result produced by the build that created the build map. This then provides access to build logs, error messages, saved build properties as well as the build report attachment files for the build. You can also optionally provide a description for this build map.

Sources

Sources is a list of SCM repositories involved in building the program. They are also referenced by resolved Inputs and Binary Inputs in order to reduce metadata duplication.

SOURCE:
   ref:      MortgageApplication
   type:     GIT   
   url:      git@github.com:builder/MortgageApplication.git   
   branch:   main
   location: /u/builder/jenkins/workspace/MortgageApplication
   snapshot: 87347e8fa01351f6fea5e0dc6e03284e971a63f8

Populating sources

While you can manually add sources to the build map by using the addSource method, you can easily add Git source repositories by using the populateGitSources method.

// populate Git sources by providing a list of local Git clone directories to introspect
List<File> repos = new ArrayList<File>();
repos.add(new File("/u/builder/jenkins/workspace/MortgageApplication"));
buildMap.populateGitSources(repos);

Another option to populate the sources list with Git repositories is to use the Git aware populateInputsFromGit method described below. This method also populates the sources list when identifying input source files.

Inputs

Inputs is a list of source files that are used to build the program. This list includes both the program source file (marked category: SELF) as well as resolved and unresolved build dependencies provided by source code scanners and resolved by the DBB dependency resolver.

INPUT:
   lName:    EPSCMORT
   category: SELF
   path:     MortgageApplication/cobol/epscmort.cbl
   source:   MortgageApplication
   version:  25447e8fa01351f6fea5e0dc6e03284e971a1222

INPUT:
   lName:    EPSMTCOM
   category: COPY
   path:     MortgageApplication/copybook/epsmtcom.cpy
   source:   MortgageApplication
   version:  19c37e8fa01351f6fea5e0dc6e03284e971a1aa67
   
INPUT:
   lName:    DFHAID
   category: INCLUDE

Populating inputs

Beside the addInput method, which you can use to manually add inputs, DBB also provides the populateInputsFromGit method. This method uses the SearchPathDependencyResolver class to automatically locate and retrieve Git SCM information for the build dependencies of the program.

// populate the inputs list using dependency resolution including Git SCM metadata
String repo = "/u/builder/jenkins/workspace/MortgageApplication";
String copybookSearch = "search:" + repo + "/?path=copybook/*.cpy";
buildMap.populateInputsFromGit(repo, copybookSearch);

Binary inputs

Binary inputs contains a list of statically linked subprograms that are included in the load module if the program build produces one. The DBB link-edit parser scans program modules and identifies statically linked compile units.

BINARY INPUT:
   member:   EPSNBRVL
   dataset:  BUILDER.MORTAPP.OBJ
   path:     MortgageApplication/cobol/epsnbrvl.cbl
   source:   MortgageApplication-main
   version:  7e5a7e8fa01351f6fea5e0dc6e03284e971a100ef

BINARY INPUT:
   member:   HOGAN5
   dataset:  THIRD.PARTY.LOAD

Populating binary inputs

Beside the addBinaryInput method, which you can use to manually add binary inputs, DBB also provides the populateBinaryInputsFromGit method. This method scans the output load module of the program, identifies statically linked subprograms built by DBB, and retrieves the subprogram build map to get the source file information of the program that created the binary input.

// populate the binary inputs list using by scanning this program's generated load module
buildMap.populateBinaryInputsFromGit("BUILDER.MORTAPP.LOAD", "EPSCMORT");

Outputs

Outputs contains the list of the output artifacts that were built by the program. Like the other list types, you can use the fields anyway you want. However, to address the impact analysis gap, the dataset and member fields must be provided.

Outputs can support dataset member outputs and z/OS UNIX System Services file outputs (path). A fingerprint field for storing the unique load module ID provided by the binder as well as a user-defined version field are also available.

OUTPUT:   
   member:      EPSCMORT
   dataset:     BUILDER.MORTAPP.LOAD
   deployType:  CICSLOAD
   fingerprint: 25447e8fa01

OUTPUT:   
   path:       /u/builder/mortgageApp/epscmort.wsdl
   deployType: WSDL
   version:    1678985691

Beside the addOutput method, which you can use to manually add outputs, DBB also provides the populateOutputs method. This method populates the output list from a list of DBB executable commands such as MVSExec, TSOExec, or ISPFExec.

// populate the output list with outputs derived from DBB executable command DD Statements
List<IExecute> execs = new ArrayList<IExecute>();
execs.add(compile);
execs.add(linkEdit);
buildMap.populateOutputs(execs);

Retrieving existing build maps

To retrieve an existing build map from the metadata store, use the BuildGroup getBuildMap method.

BuildGroup buildGroup = MetadataStore.getBuildGroup("MortgageApplication");
BuildMap buildMap = buildGroup.getBuildMap("MortgageApplication/cobol/epscmort.cbl");

You can also search for build maps based on the outputs of build maps. The BuildGroup findBuildMaps(member, dataset, path, deployType) method finds all the build maps in the build group that have an Output that matches all of the provided Output members. To ignore parameters during the search, pass null as the argument.

Note: The metadata store filesystem implementation is limited to searching only by member and dataset members. Passing non-null values for path and deployType will cause an exception.

// find all build maps that create an SQL DBRM
BuildGroup buildGroup = MetadataStore.getBuildGroup("MortgageApplication");
List<BuildMap> dbrmBuildMaps = buildGroup.findBuildMaps(null, "BUILDER.MORTAPP.DBRM", null, null);

// find all build maps that have a CICS deployment module
BuildGroup buildGroup = MetadataStore.getBuildGroup("MortgageApplication");
List<BuildMap> CICSBuildMaps = buildGroup.findBuildMaps(null, null, null, "CICSLOAD");