Question & Answer
Question
The TWSz DATASTORE started task may experience a SPACE ABEND, such as S878, or extremely high CPU utilization, while attempting to process a very large joblog.
Cause
Incorrect/inappropriate settings of DSTOPTS parameters MAXUNPAGES and MAXMVSPAGES, or TWSz CONTROLLER JOB-TAILORING EXIT (EQQUX013) not activated.
Answer
When the DATASTORE DSTOPTS includes parameter STORESTRUCMETHOD(DELAYED) -- as recommended by IBM for performance reasons, the datastore attempts to store the UNSTRUCTURED (human readable) joblog for every job when it ends. The STRUCTURED data (for use by Restart and CLeanup processing) is created by parsing the UNSTRUCTURED data only when a restart is actually initiated.
However, there is a physical limit to the size of JOBLOG that can be processed into UNSTRUCTURED data. This limit varies by system, so the DSTOPTS MAXUNPAGES() parameter is provided to enable the user to set a limit of his choosing. MAXUNPAGES() specifies the maximum size (in 4K pages) of joblog that the DATASTORE will attempt to capture into the UDF database. If the joblog is bigger than the setting of MAXUNPAGES, then no attempt is made to store the unstructured data. Instead, the datastore tries to create the STRUCTURED data (which is much more compact, but which requires a lot more processing) and message EQQFJKLW is written to the DATASTORE EQQMLOG:
- EQQFJKLW JOB: JOBNMID TOO LARGE - R&C IS ALLOWED BUT JOBLOG IS NOT
STORED. JOBLOG WILL BE REQUEUED TO FAILD.
IBM SUPPORT recommends an initial setting of MAXUNPAGES(512). This tells the datastore not to attempt to capture UNSTRUCTURED DATA for any joblog that exceeds 2M (2,097,152) bytes, or a bit less than 16,000 lines of sysout data.
. .
The size of joblog that can be captured into STRUCTURED DATA is much larger than what can be captured to UNSTRUCTURED DATA, but creating the STRUCTURED DATA is very CPU intensive, and an extremely large joblog may tie up the DATASTORE with high cpu usage for several minutes.
Therefore, APAR PK97986, for TWSZ 8.2, 8.3, and 8.5.0, and APAR PM05335 for TWSz 8.5.1, add a new DSTOPTS parameter MAXMVSPAGES().
.
If a joblog is bigger than the MAXMVSPAGES setting, it is requeued by the DATASTORE directly to the FAILDEST destination. No attempt is made to capture EITHER Structured or Unstructured data, and message EQQFJKNE is written to the DATASTORE EQQMLOG:
- EQQFJKNE JOB: JOBNMID MVS JOB LOG TOO LARGE. JOB LOG IS NOT STORED,
NOT PARSED AND REQUEUED TO FAILD.
MAXMVSPAGES is specified as a number of 4K "pages" exactly like MAXUNPAGES, and must always be set to a value larger than MAXUNPAGES. IBM support recommends an initial setting of MAXMVSPAGES(1024), or 4M (4,194,304) -- about 32,000 lines of sysout.
Users should check the DATASTORE EQQMLOG periodically, and if excessive numbers of either message EQQFJKLW or EQQFJKNE are received, the associated parameter should be adjusted as needed.
IBM support also recommends implementing TWSz user exit EQQUX013, to tell the controller to not add the TWSDSTxx OUTPUT statements to selected submitted jobs, and thus to not create the DATASTORE copy of the SYSTEM SYSOUT for those jobs. The IBM-supplied sample EQQUX013 provides options to filter jobs by WORKSTATION, JOBNAME, APPLICATION NAME, OPERATION TYPE (JOB or STARTED TASK), and by OPERATION CLEANUP TYPE.
Of these, the most easily implemented and effective are the OPERTYP and CLEANUPT subroutines.
There is no use creating DATASTORE output for started tasks... they're never going to be restarted. And there is no use creating DATASTORE output for jobs with CLEAN UP TYPE=NONE.
Implementing EQQUX013 and activating the OPERTYP and CLEANUPT options can greatly reduce datastore overhead and improve overall TWSz performance.
.
And correct settings of the DSTOPTS MAXUNPAGES() and MAXMVSPAGES() parameters will prevent datastore problems caused by an unusually large joblog.
Was this topic helpful?
Document Information
Modified date:
13 September 2019
UID
swg21422979