What DBB APIs issue z/OS commands

DDStatement

The DDStatement class is a utility class that is used by the three Exec commands (MVSExec, ISPFExec, TSOExec) to define data definition (DD) statements that will automatically be allocated before and freed after the program or command is run.

// Delete a dataset by calling IEFBR14
def delete = new MVSExec().pgm("IEFBR14")

def oldDS = new DDStatement()
oldDS.setName("OLDDS")
oldDS.setDsn("USR1.BUILD.COBOL")
oldDS.setOptions("old delete")
delete.addDDStatment(oldDS)

def rc = delete.execute()

// Using setter method chaining
def delete = new MVSExec().pgm("IEFBR14")
delete.dd(new DDStatement().name("OLDDS").dsn("USR1.BUILD.COBOL").options("old delete"))
def rc = delete.execute()

A DD statement is constituted of the following four main parts:

• DD Name (name) - The data definition name. Can be omitted (see DD Concatenation below).
• Data Set Name (dsn) - The data set to be allocated for the program to read or write. Can be omitted (see Temporary Data Sets below).
• Path (path) - The path name of a file to allocate. Mutually exclusive to DSN. Can be omitted.
• BPXWDYN Options (options) - The DDStatement uses the BPXWDYN text interface to dynamically allocate data sets during the build. The options attribute must consist of valid BPXWDYN dynamic allocation options.

DD Concatenation

Many z/OS compilers, linkers require one or more data sets to be concatenated together to form a library to be searched for referenced modules that are needed by the z/OS program. In JCL, this is done by creating a DD concatenation.

// JCL
//SYSLIB   DD   DSNAME=USR1.OBJ,DISP=SHR
//         DD   DSNAME=CEE.SCEELKED,DISP=SHR
//         DD   DSNAME=DFH.CICS.SDFHLOAD,DISP-SHR

A DD concatenation can be defined in two ways in DBB:

• Create a parent DDStatement and manually concatenate additional DDStatements to it.

  def linker = new MVSExec().pgm("IEWBLINK")
  def syslib = new DDStatement().name("SYSLIB").dsn("USR1.OBJ").options("shr")
  syslib.concatenate(new DDStatement().dsn("CEE.SCEELKED").options("shr"))
  syslib.concatenate(new DDStatement().dsn("DFH.CICS.SDFHLOAD").options("shr"))
  linker.dd(syslib)
  

• Adding a DDStatement without a DD name to a MVSExec command will automatically concatenate it to the last DDStatement added with a DD name.

  def linker = new MVSExec().pgm("IEWBLINK")
  linker.dd(new DDStatement().name("SYSLIB").dsn("USR1.OBJ").options("shr"))
  linker.dd(new DDStatement().dsn("CEE.SCEELKED").options("shr"))
  linker.dd(new DDStatement().dsn("DFH.CICS.SDFHLOAD").options("shr"))
  

TASKLIB vs STEPLIB

A major difference between the JCL EXEC statement and the DBB MVSExec command is that the JCL STEPLIB has been replaced with a TASKLIB in DBB. This is because DBB runs as a Java application in a zOS UNIX subsystem where the STEPLIB is defined as an environment variable that cannot be modified during the DBB build process. Additionally, DBB invokes z/OS programs by using the ATTACHX assembler macros. The ATTACHX macro uses the TASKLIB as its search path to locate the program being invoked.

// JCL
//COBOL  EXEC PGM=IGYCRCTL,REGION=0M,PARM='LIB'                                                                                    
//STEPLIB  DD DISP=SHR,DSN=IGY.SIGYCOMP                                                                           

// DBB
def cobol = new MVSExec().pgm("IGYCRCTL").parm("LIB")
cobol.dd(new DDStatement().name("TASKLIB").dsn("IGY.SIGYCOMP").options("shr"))

Temporary data sets

Temporary data sets are data sets that are created when they are allocated and deleted when they are freed. In DBB, a temporary data set is created by defining a DDStatement with no data set name (DSN) or providing a data set name that begins with "&&".

// Create temporary datasest that exists only for the length of the MVSExec command
compile.dd(new DDStatement().name("SYSUT1").options("cyl space(5,5) unit(vio) new"))
compile.dd(new DDStatement().name("SYSLIN").dsn("&&TEMPOBJ").options("cyl space(5,5) unit(vio) new"))

Passing temporary data sets

In some cases, it is useful to have a temporary data set continue to exist after the MVSExec command completes to be used by another MVSExec command. To do so, use the pass option when declaring the DDStatement, which will prevent the MVSExec command from freeing the DD statement upon completion.

compile.dd(new DDStatement().name("SYSLIN").dsn("&&TEMPOBJ").options("cyl space(5,5) unit(vio) new").pass(true))

After the compile MVSExec command completes, the SYSLIN DD allocation still exists and can be used by following MVSExec commands such as the linker which requires a SYSLIN allocation as input.

def linkedit = new MVSExec().pgm("IEWBLINK").parm("MAP,RENT,COMPAT(PM5)")

The pass option only applies to DD statements with temporary data sets. When you use the pass option, you need to provide a data set name for temporary data sets. For permanent data sets, it is ignored if set and the DD allocation will be freed at the end of the MVSExec command as normal.

IMPORTANT Passed DD statements can cause a problem if not used in conjunction with the MVSJob utility class.

Free temporary DDs allocated by REXX and C compilers

Some compilers such as REXX and C often allocate temporary DDs when certain files are being included. These temporary DDs are not freed automatically at completion of the compiler run. Beginning from DBB V1.0.3, MVSExec has a freePgmAllocatedDDs() option that you can use to free these DDs when a compiler run completes. The freePgmAllocatedDDs() option triggers a comparison of all the DDs before and after the compiler is called. If some DDs are detected to be not freed by the compiler, specifying the freePgmAllocatedDDs() option frees those DDs.

MVSJob

In a JCL, a passed DD statement is automatically freed at the end of the JCL job. However, in DBB this would not happen until the entire build process finishes. This can cause a problem when the same called build script is invoked repeatedly while building a list of programs of the same type. The second time the script is invoked, it would fail due to a BPXWDYN allocation error. Probably either a 0x410 - Specified ddname unavailable or a 0x420 - Specified ddname or dsname associated with an open data set.

DBB provides MVSJob, which frees DDs until the end of MVSJob, so that you can use MVSJob to define the scope of the passed DD statement. This is useful with temporary data sets. Moreover, you can embed several MVSExec commands in one MVSJob.

// use MVSJob start and stop commands to handle passed DDs
def job = new MVSJob()
job.start()
rc = compile.execute()
if (rc <= 4)
    rc = linkedit.execute()
job.stop()

When the MVSJob start method is invoked, all the passed DDStatements with temporary data sets are registered in a table when allocated and will remain allocated until the MVSJob stop method is invoked, in which case all of the DDStatements in the table are freed. There is no limit or restriction of how many or what type of statements can be coded between the MVSJob start and stop method invocations. All statements are ignored except MVSExec statements.

MVSJob also has an executable mode that uses setter method chaining to add several MVSExec commands to the MVSJob and then execute the entire MVSJob. An explicit start() is invoked at the beginning of the job execution and an implicit stop() is invoked at the end of the job.

// Build a BMS map
def copybookGen = new MVSExec().file(file).pgm("ASMA90").parm("SYSPARM(DSECT),DECK,NOOBJECT")
. . .
def compile = new MVSExec().file(file).pgm("ASMA90").parm("SYSPARM(MAP),DECK,NOOBJECT")
. . .
def linkedit = new MVSExec().file(file).pgm("IEWBLINK").parm("MAP,RENT,COMPAT(PM5)")
. . .
def rc = new MVSJob().executable(copybookGen).executable(compile).executable(linkedit).maxRC(0).execute()

In the above example, the three MVSExec commands are executed in the order they are added to the MVSJob. An optional maxRC can be added that will cause the job to terminate early if any MVSExec command returns an RC greater than the maxRC.

Multi-thread support with MVSJob

Executing two or more MVSExec commands at the same time in a multi-thread environment often requires the ability to concurrently allocate the same DD name. This is generally not possible when running the commands in a single JVM because they use the same address space and will often lead to an error when a second process tries to allocate the same DD that was already allocated by the first process. Beginning with v1.0.2, DBB provides a new Java option DBB_SUBPROCESS_ENABLED that when set to true will cause all MVSExec executions and DD allocations performed as part of a MVSJob to be done in a separate address space. The DBB_PROCESS_ENABLED option can be set using Java -D option:

groovyz -DDBB_SUBPROCESS_ENABLED=true build.groovy

Note: You might have set the USS environment in such a way as to prevent processes from creating separate address spaces. While this is not an issue when launching DBB build scripts from a Jenkins remote agent, it does impact DBB build processes launched from the command line. It is recommented to use the DBB groovyz shell script included in the DBB toolkit at /usr/lpp/IBM/dbb/bin/groovyz to launch DBB build scripts from the command line as it has been modified to allow for the creation of additional address spaces.

Copying temporary data sets

As mentioned in the section covering the CopyToHFS command, it can be useful to copy the SYSPRINT output from compilers and linkers to zFS as log files to retain for later use. Instead of having to write the SYSPRINT file to a permanent data set, the CopyToPDS can also use as its source a DD statement name. Additionally declared CopyToHFS commands can be added to an MVSExec command in the same way the DD statements are added. These CopyToHFS commands are then executed as part of the MVSCommand execution.

def compile = new MVSExec().file(file).pgm("IGYCRCTL").parm("LIB")
compile.dd(new DDStatement().name("SYSPRINT").options("cyl space(5,5) unit(vio) new"))
compile.copy(new CopyToHFS().ddName("SYSPRINT").file(new File("/u/usr1/build/helloworld.log").hfsEncoding("UTF-8"))

When the MVSExec command is executed, the following occurs:

1. All the DD statements are allocated.
2. The z/OS program is executed.
3. All the CopyToHFS commands are executed.
4. All the DD statements are freed except the statements with temporary data sets marked as pass.

In-stream data sets

An in-stream data set is a way to define input content to a z/OS program without having to read in a file or data set from disk.

// JCL
//SYSIN *
HELLO
/*

// DBB 
new DDStatement().name("SYSIN").instreamData("HELLO")

Use Java newline characters to separate the String into multiple lines of text.

// JCL
//SYSIN *
RECORD1
RECORD2
RECORD3
/*

// DBB
def records = "RECORD1\nRECORD2\nRECORD3"
new DDStatement().name("SYSIN").instreamData(records)

You can also use triple quotes for this purpose:

// JCL
//SYSIN *
RECORD1
RECORD2
RECORD3
/*

// DBB
def records = '''RECORD1
RECORD2
RECORD3'''
new DDStatement().name("SYSIN").instreamData(records)

Note: In-stream data has a maximum line length of 80 characters. It is typical that column 1 for In-stream data would be a white space.

Using ISPF Gateway

def memberCopy = new TSOExec().command("OCOPY INDD(IN) OUTDD(OUT) TEXT CONVERT(YES) TO1047")
memberCopy.confDir("/usr/lpp/IBM/dbb/conf")

# The dbb.gateway.type property determines which gateway type is used for the entire build process
# Possible values are 'legacy' and 'interactive.  Default if not indicated is 'legacy'
dbb.gateway.type=interactive

def memberCopy = new TSOExec().command("OCOPY INDD(IN) OUTDD(OUT) TEXT CONVERT(YES) TO1047")
memberCopy.gatewayType("interactive")

// legacy gateway commands require DD allocation CMDSCP
def memberCopy = new TSOExec().command("OCOPY INDD(IN) OUTDD(OUT) TEXT CONVERT(YES) TO1047")
memberCopy.dd(new DDStatement().name("CMDSCP").dsn("USR1.ISPFGWY.EXEC").options("shr"))
memberCopy.dd(new DDStatement().name("IN").dsn("USR1.BUILD.LOAD(EPSCMORT)").options("shr"))
memberCopy.dd(new DDStatement().name("OUT").dsn("USR1.BUILD.LOADBK(EPSCMORT)").options("shr"))
memberCopy.logFile(new File("/u/usr1/build/logs/EPSCMORT_ispf.log"))
int rc = memberCopy.execute();

// HELLO Rexx script keeps prompting for a name
def hello = new TSOExec().command("exec 'USR1.REXX(HELLO)'")
hello.execute()
println hello.getOutput()

["John", "Paul", "George", "Ringo"].each { name ->
   if (hello.isWaitingForResponse()) {
       println name
       hello.response(name).execute()
       println hello.getOutput()
   }
}

// if the hello program is still waiting for a response then cancel the session to end
if (hello.isWaitingForResponse()) {
      println "Cancelling conversation"
      hello.cancel()
}

What is your name?
John
Hello John
What is your name?
Paul
Hello Paul
What is your name?
George
Hello George
What is your name?
Ringo
Hello Ringo
What is your name?
Cancelling conversation

# Procedure Name - specified with the procname parameter
dbb.gateway.procedureName=

# Account number - specified with the acctnum parameter
dbb.gateway.accountNumber=

# Group name - specified with a groupid parameter
dbb.gateway.groupId=

# Region size - specified with the regionsz parameter
dbb.gateway.regionSize=

# Gateway logging level.  Add values for multiple types:
# 1 - Log error information
# 2 - Log debug information
# 4 - Log communication information
# 8 - Log time information
# 16 - Log information to the system console
dbb.gateway.logLevel=2

Using ISPF gateway in multi-threaded mode

By default, DBB runs its build process in a single thread and therefore builds programs one at a time. This is not an issue for incremental builds or even full builds for small applications. However, when an application is composed of hundreds of programs, it causes a full build to take hours to run. You can speed up the build process by setting up a multi-threaded build; therefore, multiple programs can build at the same time thus speeding up a full build process significantly.

To use TSOExec or ISPFExec to call the ISPF gateway in multi-threaded mode, complete the following steps:

1. Copy ISPZXENV to a new location (such as /etc/ispf)
2. Update the settings in ISPZXENV for CGI_ISPPREF
3. Add the location to your runIspf.sh PATH
4. If necessary, update DBB_CONF or confDir to point to the modified runIspf.sh directory

Additional details can be found in the following sections.

/* Running multithreaded we get issues with ISPF.VCMISPF not being unique */ 
/* Use a hex representation of the PID to uniquely create ISPF.VCMISPF */ 

address syscall 'getpid' ; pid = retval                                      

Say 'Pid:'pid                                                                

If Length(Pid) > 7 Then                                        
Do                                                                           
  CGI_ISPPREF = '&SYSPREF..P$'D2X(Substr(pid,1,5))||,
                         '.P$'D2X(Substr(pid,6))
End                                                                          
Else                                                                         
  CGI_ISPPREF = '&SYSPREF..P'D2X(pid)