How to create and access a DBB build report

IBM Dependency Based Build (DBB) provides APIs to create and generate a build report which contains summary information of the build commands executed during the build. The build report is generated as a JSON file on the local file system at the end of the build and can be rendered as an HTML file that can be custom formatted and viewed in a browser.

Image of the default rendered build report provided by DBB

Build report design

At its most basic level the build report is simply a collection of records each of which contain information about a build command that was executed during the build. These records are automatically created and added to the build report when the command is executed. This assumes that the build report exists when the command is executed. If the build report has not been created when the command is executed, then no record is created or stored. Each command creates a specific type of record to be added to the build report.

* All record types inherit from the base Record class.
** ExecuteRecord and CopyToPDSRecord build reporting attributes

Initializing and accessing the build report

At the beginning of the DBB build process, the user can use the BuildReportFactory static method createDefaultReport() to create a BuildReport instance. The factory maintains a single static instance of the BuildReport that exists for the entire build process. Once the BuildReport instance is created, the user can directly access it by using the BuildReportFactory static method getBuildReport().

// During build initialization
BuildReportFactory.createDefaultReport()
. . .
// To access the BuildReport during the build
def buildReport = BuildReportFactory.getBuildReport()

Setting ExecuteRecord build report attributes

In the record types table shown above most the attributes shown for each record type are populated automatically by the DBB command class generating the record based on command API arguments. However in the case of the ExecuteRecord, the attributes marked with '**' are not required by the execute command APIs for execution. They are there for capturing useful information specifically for displaying in the build report. As such the user will need to explicitly add the attribute information using optional command API arguments.

File attribute

The file attribute refers to the source file being built. It is used by the default DBB rendering script as the key when collecting ExecuteRecords to display together showing all of the commands executed to build a source file. This attribute is part of the MVSExec, ISPFExec, TSOExec, and JCLExec command classes.

// Build a BMS Map
def copybookGen = new MVSExec().file("MortgageApplication/bms/epsmlis.bms").pgm("ASMA90").parm("SYSPARM(DSECT),DECK,NOOBJECT")
. . .
def compile = new MVSExec().file("MortgageApplication/bms/epsmlis.bms").pgm("ASMA90").parm("SYSPARM(MAP),DECK,NOOBJECT")
. . .
def linkedit = new MVSExec().file("MortgageApplication/bms/epsmlis.bms").pgm("IEWBLINK").parm("MAP,RENT,COMPAT(PM5)")

In the example above it takes three commands to build a BMS map. By setting the file() argument on each command to the same file, the ExecuteRecord for each command can be collected for file MortgageApplication/bms/epsmlis.bms and displayed together in the build report HTML file as shown in the beginning of this document.

Data sets attribute

The data sets attribute is a list of data sets used by the execute command. It is set by the DDStatement utility class by adding a report(true) attribute to a DDStatement that has the dsn attribute set. While any DDStatement used by the execute command can be flagged as a report dataset, the MortgageApplication sample application only flags input datasets in its build report.

// Flag input datasets for inclusion in the build report
compile.dd(new DDStatement().name("SYSIN").dsn("$cobolPDS($member)").options("shr").report(true))

Outputs attribute

The outputs attribute is a list of datasets that have been marked as the output of a execute command. It is set by the DDStatement utility class by adding a output(true) attribute to a DDStatement that has the dsn attribute set.

// Flag output datasets for inclusion in the build report
linkedit.dd(new DDStatement().name("SYSLMOD").dsn("$loadPDS($member)").options("shr").output(true))

DeployType attribute

The deployType attribute is an additional attribute of a data set marked as an output data set (see Outputs attribute above). It is designed to be used primarily by post build processing tools such a UrbanCode Deploy that deploy build artifacts to production systems. It is set by the DDStatement utility class by adding a deployType attribute to a DDStatement that has the output(true) attribute set. It is only valid when used with a DDStatement flagged as output(true) and is ignored otherwise. The value of the deployType attribute can be any valid String value required by the post build processing tool.

// Provide a deploy type for output datasets that can be used in post build tools like UrbanCode Deploy
linkedit.dd(new DDStatement().name("SYSLMOD").dsn("$loadPDS($member)").options("shr").output(true).deployType("LOAD"))

LogFiles attribute

The logFiles attribute is a list of zFS files created and/or updated by the execute command. These files are created/updated by the CopyToHFS commands added to the execute command and are assumed to be SYSPRINT files copied to zFS as log files. This attribute is set automatically for each CopyToHFS command added to the execute command.

// Copy subcommands automatically contribute to the logFiles attribute
compile.copy(new CopyToHFS().ddName("SYSPRINT").file(logFile))

Setting CopyToPDSRecord build report attributes

Another common build step is copying the JCL file required to execute the built binary. Similar to the ExecuteRecord, you can indicate that a CopyToPDS command should contribute to the build report and display not just source and destination but also the deployment type so that data sets can be correctly handled by an automated deployment manager like UrbanCode Deploy.

Key attribute

The key attribute refers to the source file being built. It is used by the default DBB rendering script as the key when DBB collects ExecuteRecords and CopyToPDSRecords to display together to show all of the commands executed to build a source file.

new CopyToPDS().file(new File("${properties.sourceDir}/$file")).dataset(cobolPDS).member(member).key(file).execute()

Outputs attribute

The outputs attribute contains the target data set when the CopyToPDS command has been flagged as an output for the build report. To set it, add an output(true) attribute to the CopyToPDS command.

NOTE: The output(true) option is required to generate build report attributes to the CopyToPDSRecord. If output(true) is missing from the command, the other build report attributes are not generated.

// Flag the target data set for inclusion in the build report
new CopyToPDS().file(new File("${properties.sourceDir}/$file")).dataset(cobolPDS).member(member).output(true).key(file).execute()

DeployType attribute

The deployType attribute is an additional attribute of the target data set. It is used primarily by post build processing tools such as UrbanCode Deploy that deploy build artifacts to production systems. The value of the deployType attribute can be any valid String value required by the post build processing tool.

// Provide a deploy type for the target dataset that can be used in post build tools like UrbanCode Deploy
new CopyToPDS().file(new File("${properties.sourceDir}/$file")).dataset(cobolPDS).member(member).output(true).key(file).deployType("JCL").execute()

User-defined record type

All of the build report record types discussed so far are generated automatically by DBB command APIs and are designed with specific attributes for the command that generates them. In order to allow users to create and add custom records to the build report manually, DBB provides a PropertiesRecord record type. The PropertiesRecord is a simple record that contains an internal properties table in which users can add their own property names and values as record attributes. The PropertiesRecord is in the com.ibm.dbb.build.report.records package.

// Add a build report record containing build information
def buildInfo = new PropertiesRecord("buildInfo")
buildInfo.addProperty("group", "module_1")
buildInfo.addProperty("label", "build-${new Date()}")
buildInfo.addproperty("description", "Online processing module #1")

BuildReportFactory.getBuildReport().addRecord(buildInfo)

Generating and rendering build report

The DBB build report exists only in memory while the DBB build process is running and will cease to exist once the build process terminates. At the end of the build before the process terminates, the user must save the build report to a file on the local file system.

// get the build report
def buildReport = BuildReportFactory.getBuildReport()

// create build report data file
def jsonOutputFile = new File("usr1/build/BuildReport.json")
def buildReportEncoding = "UTF-8"
buildReport.save(jsonOutputFile, buildReportEncoding)

The content of the saved build report is in JSON format so the convention is to add a .json file extension to the save file name. This file is also sometimes referred to as the build report data file.

Sample content of build report JSON file (formatted)

{"records":[
  {"type":"VERSION","id":"VERSION_1","date":"28-Feb-2018 21:58:25","build":"145","version":"1.0.0"},
  {"type":"BUILD_RESULT","id":"BUILD_RESULT_1","label":"build.20180305.075907.059","url":"https:\/\/ee-jenkins.rtp.raleigh.ibm.com:9443\/dbb\/rest\/buildResult\/5140","group":"MortgageAppUser1sTVT6031"},
  {"type":"COPY_TO_PDS","id":"COPY_TO_PDS_1","destination":"USER1.MORTAPP.BMS(EPSMLIS)","source":"\/u\/user1\/test\/jenkins\/workspace\/User1sMortgageApp\/MortgageApplication\/bms\/epsmlis.bms"},
  {"type":"EXECUTE","id":"EXECUTE_1","file":"MortgageApplication\/bms\/epsmlis.bms","command":"ASMA90","options":"SYSPARM(DSECT),DECK,NOOBJECT","rc":0,"outputs":[{"id":"USER1.MORTAPP.COPYBOOK(EPSMLIS)","type":"OUTPUT","dataset":"USER1.MORTAPP.COPYBOOK(EPSMLIS)"}],"datasets":["USER1.MORTAPP.BMS(EPSMLIS)"],"logs":["\/u\/user1\/test\/jenkins\/workspace\/User1sMortgageApp\/BUILD-9\/EPSMLIS.log"]},
  . . .

Once the build report data file is saved, a second HTML file needs to be created that when loaded in a web browser will transform the content of the data file into the build report image shown at the beginning of this document. DBB provides an HTMLTransformer utility class that will generate the HTML file.

Note: The default generated HTML file requires the build report JSON data file to exist in the same location as the generated HTML file and that it has read access to the file.

// create a default build report html file
def htmlOutputFile = new File("usr1/build/BuildReport.html")
def htmlTemplate = null  // Use default HTML template.
def css = null       // Use default theme.
def renderScript = null  // Use default rendering.                       
def transformer = HtmlTransformer.getInstance()
transformer.transform(jsonOutputFile, htmlTemplate, css, renderScript, htmlOutputFile, buildReportEncoding)

To customize the content and appearance of the build report HTML file, users can substitute some or all of the three input files:

• HTML Template
• Cascading Style Sheet (CSS)
• JavaScript rendering function

Examples of the content of the default DBB files can be found in Default build report transformation files.

Using the build report in post-build steps

While the main purpose of the build report it to provide a nice formatted HTML summary of the build for people to look at, the build report JSON data file also allows post build processes a convenient way to obtain information about the preceding build. Additionally users can pass specific information required by the post build process via the build report by creating and adding custom PropertiesRecords. Use the BuildReport.parse method to parse a build report JSON file for post build processing.

// read build report data
def buildReportDataFile = new File("user1/build/BuildReport.json")
def buildReport= BuildReport.parse(new FileInputStream(buildReportDataFile))

// parse build report to find the build outputs to be deployed. 
// the following example finds all the build output with deployType set
def executes = buildReport.getRecords().findAll{ record ->
    record.getType()=="EXECUTE" && 
    !record.getOutputs().findAll{ o -> 
        o.deployType() != null
    }.isEmpty()
}