CARLa may be a highly efficient data analysis tool, but it lacks in the area of flow control. If you have some CARLa that should run on the first of the month, you cannot program that selection in CARLa. With zSecure 2.3.0, a newlist type RUN was introduced that contains information about the current job environment, such as the date and time, the system where the CARLa job runs, etc. However, this does not offer direct control of the reports within the job, RUN fields are meant to be used for output. To address this limitation, a data generator program was written to generate CARLa, TSO commands or arbitrary data records, using a flow control language similar to ISPF file tailoring skeletons.
DATAGEN is a Rexx program that reads commands from the command parameter, and from SYSIN. To run this Rexx, you use JCL to creates a TSO/E environment in a batch job:
//GENDATA EXEC PGM=IKJEFT1A,PARM='%DATAGEN' //SYSEXEC DD DISP=SHR,DSN=IBMUSER.REXXLIB //SYSTSIN DD DUMMY //SYSTSPRT DD SYSOUT=* //SYSIN DD * )file report today is &dd &mmm and that is julian &YYDDD yesterday was &mmdd-1 a month ago was &yymmdd-30 two months ago the month was &mm-2 and with a year in front: &yyyymm-2 //REPORT DD SYSOUT=*
Output of this job is an unimpressive report:
today is 17 FEB and that is julian 16048 yesterday was 0216 a month ago was 160118 two months ago the month was 12 and with a year in front: 201512
The )FILE command indicates where output should be generated. All control commands for DATAGEN start with a ) in position 1. Spaces between the ) and the command are allowed. Lines without a ) are copied to the selected output file, after variable substitution. Built-in variables are:
&DDD julian day number &YYYYDDD julian date &YYDDD short julian date &YYYYMMDD standard date format &YYMMDD short date format &MMDD month and day &DD day &YYMM year and month &MM month number &MMM month name &YYYY year &YY year
You can add an offset behind the variable name, such as, &YYMM-2 and DATAGEN calculates the year and month value, 2 months before this month. DATAGEN was not designed to predict the future, offset only goes back in time. However, a )DATE command is available to simulate running on another date. It takes a literal value in Rexx standard date format:
)DATE 2015-12-13
In addition, system symbols can be used as defined in IEASYMxx members and displayed with the D SYMBOLS command:
//GENDATA EXEC PGM=IKJEFT1A,PARM='%DATAGEN DATE 2015-12-31' //SYSEXEC DD DISP=SHR,DSN=IBMUSER.REXXLIB //SYSTSIN DD DUMMY //SYSTSPRT DD SYSOUT=* //SYSIN DD * )file report Running on &sysname, sysclone &sysclone, sysplex &sysplex Today is &mm/&dd/&yyyy //REPORT DD SYSOUT=*
Variable names are delimited by the following characters:
.,/:;'()
When a variable ends at a point (.) the point is removed from the output line. Other delimiters remain in the output stream.
DATAGEN can be used to generate reports from the catalog, using filters similar to the ones in ISPF option 3.4. When the parameter of the )DO command contains a generic character, i.e., a * or a %, the pattern is used as a catalog search argument. Also, you can use variables in the search argument:
//GENDATA EXEC PGM=IKJEFT1A,PARM='%DATAGEN' //SYSEXEC DD DISP=SHR,DSN=IBMUSER.REXXLIB //SYSTSIN DD DUMMY //SYSTSPRT DD SYSOUT=* //SYSIN DD * )file report data sets for this system &sysname )do systemnames=sys1.&sysname..** &systemnames )enddo //REPORT DD SYSOUT=* //
Every data set name found in the catalog results in an iteration of the DO loop, and references to the loop control variable generate output lines. The list does not distinguish between the entry types, so you see clusters, components, aliases and more:
data sets for this system ZT01 SYS1.ZT01.APPCSI SYS1.ZT01.APPCSI.DATA SYS1.ZT01.APPCSI.INDEX SYS1.ZT01.APPCTP SYS1.ZT01.APPCTP.DATA SYS1.ZT01.APPCTP.INDEX SYS1.ZT01.AUTOCKPT SYS1.ZT01.AUTOCKPT.DATA SYS1.ZT01.AUTOCKPT.INDEX SYS1.ZT01.COMMON SYS1.ZT01.COMMON.DATA SYS1.ZT01.COMMONN SYS1.ZT01.COMMONN.DATA
The same mechanism could be used to scan the catalog for data set names that are known to contain critical information, and generate SIMULATE SENSITIVE commands for these. As documented, SIMULATE SENSITIVE and SIMULATE CLASS=DATASET do not support masks, but require fully qualified data set names, so if we can rely on the local catalog instead of CKFREEZE, we can use DATAGEN to generate these for us and feed the result into CKRCARLA:
//GENDATA EXEC PGM=IKJEFT1A,PARM='%DATAGEN'
//SYSEXEC DD DISP=SHR,DSN=IBMUSER.REXXLIB
//SYSTSIN DD DUMMY
//SYSTSPRT DD SYSOUT=*
//SYSIN DD *
)file report
Found PCI data sets:
)do pcidsn=BANKPROD.**.VISA.*
&pcidsn
)file carla
simulate class=dataset access=read sensitivity=PCI-PAN,
resource=&pcidsn
)enddo
//REPORT DD SYSOUT=*
//CARLA DD DISP=(NEW,PASS),SPACE=(CYL,(1,1)),DSN=&CARLA
//*
//CKRCARLA EXEC C2RC
//CARLA DD DISP=(OLD,DELETE),DSN=&CARLA
//SYSIN DD *
alloc type=racf active
alloc type=ckfreeze dsn=ibmzsec.data.szt01.ckfreeze
include dd=carla /* simulate commands */
newlist type=trusted,
title="Access to VISA card data"
define number_users sumcount
select sensitivity=pci-pan
summary racf_profile * resource number_users * userid(nd)
summary racf_profile * userid count(nd)
//
DATAGEN generates a report:
Found PCI data sets: BANKPROD.DAILY.TRANS.VISA.AUTHDATA BANKPROD.DAILY.TRANS.VISA.PANDATA BANKPROD.MONTHLY.TRANS.VISA.PANDATA
and SIMULATE commands that help CKRCARLA attribute sensitivity to these data sets. The resulting report from CKRCARLA looks like:
T R U S T E D U S E R S A N D R E S O U R C E S 17 Feb 2016 01:00 page 1
Access to VISA card data
Profile Resource NUMBER_U
BANKPROD.** 3
BANKPROD.DAILY.TRANS.VISA.AUTHDATA 247
BANKPROD.DAILY.TRANS.VISA.PANDATA 247
BANKPROD.MONTHLY.TRANS.VISA.PANDATA 247
T R U S T E D U S E R S A N D R E S O U R C E S 17 Feb 2016 01:00 page 2
Access to VISA card data
Profile Userid
BANKPROD.**
$ITSP01
$ITSP02
$MISSING
AARONP
APPC
ASCH
AU
AYMERIC
BARKERL
BBONNEL
Note: you still need a CKFREEZE because the SIMULATE command will only attribute sensitivity to an existing data set.
If you do not like the standard include members of C2RC, you can also use a skinny call to CKRCARLA:
//GENDATA EXEC PGM=IKJEFT1A,PARM='%DATAGEN'
//SYSEXEC DD DISP=SHR,DSN=IBMUSER.REXXLIB
//SYSTSIN DD DUMMY
//SYSTSPRT DD SYSOUT=*
//SYSIN DD *
)file report
Found PCI data sets:
)do pcidsn=BANKPROD.**.VISA.*
&pcidsn
)file carla
simulate class=dataset access=read sensitivity=PCI-PAN,
resource=&pcidsn
)enddo
//REPORT DD SYSOUT=*
//CARLA DD DISP=(NEW,PASS),SPACE=(CYL,(1,1)),DSN=&CARLA
//*
//CKRCARLA EXEC PGM=CKRCARLA,REGION=0M
//STEPLIB DD DISP=SHR,DSN=&CPREFIX..SCKRLOAD
//CARLA DD DISP=(OLD,DELETE),DSN=&CARLA
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
alloc type=racf active
alloc type=ckfreeze dsn=ibmzsec.data.szt01.ckfreeze
include dd=carla /* simulate commands */
newlist type=trusted,
title="Access to VISA card data"
define number_users sumcount
select sensitivity=pci-pan
summary racf_profile * resource number_users * userid(nd)
summary racf_profile * userid count(nd)
//
The )DO command can also take distinct values. Suppose you have 8 LPARs that all run C2PACMON. Now you want to allocate the daily files for yesterday. The DATAGEN commands look like:
)file carla )do lpar=a,b,c,d,k,l,m,p alloc type=access dsn=IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&yymmdd-1 )enddo
and the result:
alloc type=access dsn=IBMZSEC.DATA.SSYSA.C2PACMON.D160216 alloc type=access dsn=IBMZSEC.DATA.SSYSB.C2PACMON.D160216 alloc type=access dsn=IBMZSEC.DATA.SSYSC.C2PACMON.D160216 alloc type=access dsn=IBMZSEC.DATA.SSYSD.C2PACMON.D160216 alloc type=access dsn=IBMZSEC.DATA.SSYSK.C2PACMON.D160216 alloc type=access dsn=IBMZSEC.DATA.SSYSL.C2PACMON.D160216 alloc type=access dsn=IBMZSEC.DATA.SSYSM.C2PACMON.D160216 alloc type=access dsn=IBMZSEC.DATA.SSYSP.C2PACMON.D160216
To process the last three days, your DATAGEN commands would look like:
)file carla )do lpar=a,b,c,d,k,l,m,p alloc type=access dsn=IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&yymmdd-3 alloc type=access dsn=IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&yymmdd-2 alloc type=access dsn=IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&yymmdd-1 )enddo
or you could use nested loops like so:
)file carla )do lpar=a,b,c,d,k,l,m,p )do day=&yymmdd-3, &yymmdd-2, &yymmdd-1 alloc type=access dsn=IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&day )enddo )enddo
Loop control could also be used to break-up work into smaller chunks. For example, run a large data reduction as 4 smaller parts:
//GENDATA EXEC PGM=IKJEFT1A,PARM='%DATAGEN' //SYSEXEC DD DISP=SHR,DSN=IBMUSER.REXXLIB //SYSTSIN DD DUMMY //SYSTSPRT DD SYSOUT=* //SYSIN DD * )do chunk=0,1,2,3 )file carla&chunk )do lpar=a,b,c,d,k,l,m,p alloc type=access dsnpref=IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&yymm.&chunk )enddo alloc type=output dd=c2pacmon dsn=IBMZSEC.DATA.PLEX.C2PACMON.D&yymm.&chunk include dd=ckrcarla m=c2pamcmp )enddo //CARLA0 DD DISP=(NEW,PASS),SPACE=(CYL,(1,1)),DSN=&CARLA0 //CARLA1 DD DISP=(NEW,PASS),SPACE=(CYL,(1,1)),DSN=&CARLA1 //CARLA2 DD DISP=(NEW,PASS),SPACE=(CYL,(1,1)),DSN=&CARLA2 //CARLA3 DD DISP=(NEW,PASS),SPACE=(CYL,(1,1)),DSN=&CARLA3 //* //CKRCARLA EXEC C2RC //SYSIN DD DISP=(OLD,DELETE),DSN=&CARLA0 //CKRCARLA EXEC C2RC //SYSIN DD DISP=(OLD,DELETE),DSN=&CARLA1 //CKRCARLA EXEC C2RC //SYSIN DD DISP=(OLD,DELETE),DSN=&CARLA2 //CKRCARLA EXEC C2RC //SYSIN DD DISP=(OLD,DELETE),DSN=&CARLA3
The )IF command may be used to selectively generate code. For example, to consolidate ACCESS files from all your LPARs into one monthly, you could use:
)* monthly consolidation on the 1st of the month )if &dd=1 )file carla alloc type=output dd=c2pacmon dsn=IBMZSEC.DATA.PLEX.C2PACMON.M&yymm-1 )do lpar=a,b,c,d,k,l,m,p alloc type=access dsnpref=IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&yymm-1 )enddo include dd=ckrcarla m=c2pamcon )else /* dummy carla to prevent syntax error */ newlist type=system; summary system )endif
)IF supports the following comparison operators, the symbols may be used with or without blanks separating them from the comparands:
= <> /= \= != ¬= < <= > >= EQ NE NQ LT LE GT GE
The equal and not-equal operators support lists of values separated by commas. This example generates a message if a field is not a, b, c or &sysclone:
)if &z <> a,b,c,&sysclone z is not one of the 4 expected values: &z )endif
The )IF command may also be used to test if a data set is defined, or missing. This would check if all daily access monitor data sets were created:
)do lpar=a,b,c,d,k,l,m,p )if missing IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&yymmdd-1 )file report Yesterday's access monitor data missing for SYS&lpar )endif )enddo
You could also trigger your consolidation on the fact that the monthly data set is not there, and delete the (old) dailies after 5 days, but only if the monthly was found:
)* monthly consolidation on the 1st of the month
)set monthly=IBMZSEC.DATA.PLEX.C2PACMON.M&yymm-1
)if missing &monthly
)tso
alloc da('&monthly') +
new catlg cyl space(600 100) +
recfm(v b) lrecl(584) blksize(27998)
)file carla
alloc type=output dd=c2pacmon dsn=&monthly
)do lpar=a,b,c,d,k,l,m,p
alloc type=access dsnpref=IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&yymm-1
)enddo
include dd=ckrcarla m=c2pamcon
)else
)* monthly exists, no need to rebuild it
/* dummy carla to prevent syntax error */
newlist type=system; summary system
)if &dd>5
)* monthly exists, delete old dailies after the 5th of the new month
)do lpar=a,b,c,d,k,l,m,p
)do daily=IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&yymm-1.*
)tso
delete '&daily'
)enddo
)enddo
)endif
)endif
Using existence of the monthly for last month as the criterion to perform a consolidation is helpful in another way too. If the consolidation step fails, or if you decide that more data reduction is needed before attempting the consolidation again, you simply delete the monthly data set, and next time your daily job runs if does not find the monthly so it attempts the consolidation again.
The sample uses the )TSO command, that batches generated TSO commands and executes them after any output files were written. So you cannot practice with this code because the commands are immediately executed. You can also allocate a work file for TSO commands, like so, and set the return code of DATAGEN to indicate that commands were generated:
/GENDATA EXEC PGM=IKJEFT1A,PARM='%DATAGEN' /SYSEXEC DD DISP=SHR,DSN=IBMUSER.REXXLIB /SYSTSIN DD DUMMY /SYSTSPRT DD SYSOUT=* /TSOCMDS DD DISP=(NEW,PASS),SPACE=(CYL,(1,1)),DSN=&TSOCMDS /SYSIN DD * )if found IBMZSEC.DATA.PLEX.C2PACMON.M&yymm-1 )if &dd>5 )* monthly exists, delete old dailies after the 5th of the new month )do lpar=a,b,c,d,k,l,m,p )do daily=IBMZSEC.DATA.SSYS&lpar..C2PACMON.D&yymm-1.* )file tsocmds delete '&daily' )setrc 1 )enddo )enddo )endif )endif //* //* Execute TSO commands //* // IF GENDATA.RC=1 THEN //TSOCMDS EXEC PGM=IKJEFT1A //SYSTSPRT DD SYSOUT=* //SYSTSIN DD DISP=(OLD,PASS),DSN=&TSOCMDS // ENDIF
If the )SETRC command is not executed, because there are no data sets to delete, the return code would be 0 and the JCL interpreter skips the 2nd step. If you do not use )SETRC and execute IKJEFT1A with an empty TSOCMDS file, the step abends on an I/O error.
Commands start with a ) in position 1 of the input line. A command may also be specified in the call parameter of DATAGEN, without a preceding ).
Input lines and output lines are truncated at position 72.
) FILE ddname
) FILE dd&n
write output until next ) FILE command or ) TSO command to this destination
) TSO
execute output until next ) FILE command as TSO commands
) DO var=value1,value2,value3,...,&var
loop control
values are separated with commas, have to be numbers, words or variables, no wild characters allowed
) DO var=hlq.qual.*.type
) DO var=hlq.sys&sysclone..**
catalog lookup
if one or more wild chars are specified, the value is taken to be a catalog pattern and results in a catalog list
) ENDDO
) IF &var=value1,value2,value3,...
) IF &var<>value1,value2,value3,...
) IF &var<value
) IF &var GT value
conditional generation of output, no wild chars or &VARs
operators are = <> /= \= != ¬= < <= > >= EQ NE NQ LT LE GT GE
) IF [EXISTS | FIND | FOUND] dataset
) IF MISSING dataset
code
) ELSE
alternative
) ENDIF
) DATE 2016-01-13
simulate another date, e.g., to rerun the jobs from last week
) SETRC value
set the job step return code
) SET &var=value
set a variable for later use in DATAGEN, values have to be numbers, words or &variables, no wild characters
)* comment
) * comment
Lines without ) in pos 1 are variable substituted and written to the ddname from the preceding ) FILE command, or queued for TSO command execution.
Download (right-click and Save Link As...):