Long-term reporting with the Postprocessor

You can use the RMF Postprocessor to generate long-term overview reports for your Spark workload. Unlike the interactive Monitor III sessions, Postprocessor reports allow you to analyze your Spark workload performance over the long term and fine tune its resource assignments. For more information about running the Postprocessor and other report options, see z/OS RMF User's Guide.

Workload Activity report

The Workload Activity report (WLMGL) presents a summarized view of a WLM-managed workload. You can select to view this report for a specific service class, report class, WLM workload, and various other types. For instance, if you place all of your Spark processes in a single service class, you can generate a Workload Activity report that tells you the actual amount of memory and CPU consumed by that workload. Figure 1 shows an example of the Workload Activity report for the ODASSC1 service class.

Figure 1. Example of the RMF Workload Activity report for a Spark service class

                                               W O R K L O A D   A C T I V I T Y                                   
                                                                                                                     PAGE  48    
      z/OS V2R2               SYSPLEX ZPETPLX2           DATE 02/13/2017           INTERVAL 30.00.057   MODE = GOAL              


                              RPT VERSION V2R2 RMF       TIME 17.29.35                                                           
                                                                                                                                 
REPORT BY: POLICY=WLMPOL01   WORKLOAD=ODASWL    SERVICE CLASS=ODASSC1     RESOURCE GROUP=ODASRG                               
                                                 CRITICAL     =NONE         HONOR PRIORITY=NO
                                                 DESCRIPTION  =Sample Spark service class
                                                                                                                                 
-TRANSACTIONS-  TRANS-TIME HHH.MM.SS.TTT  --DASD I/O--  ---SERVICE---   SERVICE TIME  ---APPL %---  --PROMOTED--  ----STORAGE----
AVG       4.19  ACTUAL               716  SSCHRT   5.5  IOC    202338   CPU  276.626  CP      0.57  BLK    0.000  AVG    167913.4
MPL       4.19  EXECUTION            715  RESP     2.3  CPU     16393K  SRB    2.816  AAPCP   0.00  ENQ    0.000  TOTAL  702887.3
ENDED      441  QUEUED                 1  CONN     2.2  MSO      1972M  RCT    0.039  IIPCP   0.14  CRM    0.000  SHARED     0.00
END/S     0.24  R/S AFFIN              0  DISC     0.0  SRB    166881   IIT    0.228                LCK    3.098                 
#SWAPS     135  INELIGIBLE             0  Q+PEND   0.1  TOT      1989M  HST    0.000  AAP      N/A  SUP    0.000  -PAGE-IN RATES-
EXCTD        0  CONVERSION             0  IOSQ     0.0  /SEC     1105K  AAP      N/A  IIP    20.85                SINGLE      0.0
AVG ENC   0.00  STD DEV            1.195                                IIP  213.360                              BLOCK       0.0
REM ENC   0.00                                          ABSRPTN   264K                                            SHARED      0.0
MS ENC    0.00                                          TRX SERV  264K                                            HSP         0.0
                                                                                                                                 
TRANSACTION APPL% :   TOTAL :   CP   0.55   AAP/IIP ON CP   0.14   AAP/IIP  20.85                                                
                      MOBILE :  CP   0.00   AAP/IIP ON CP   0.00   AAP/IIP   0.00

Table 1 describes some of the fields in the Workload Activity report that are more relevant to Spark.

Table 1. Selected fields in the RMF Workload Activity report
Field heading	Entries
SERVICE TIME	CPU Time, in seconds, consumed on general-purpose and special-purpose processors. IIT I/O interrupt time, in seconds. IIP zIIP service time, in seconds.
APPL %	CP Percentage of the processor time used by transactions running on general-purpose processors. IIPCP Percentage of the processor time used by zIIP-eligible transactions running on general-purpose processors. This is a subset of the APPL % CP value. IIP Percentage of the processor time used by transactions executed on zIIPs.
STORAGE	AVG Weighted average number of central storage frames allocated to active ASIDs. TOTAL Total number of central storage frames allocated to resident ASIDs.
PAGE-IN RATES	SINGLE Average rate at which pages are read into central storage while transactions are resident in central storage. BLOCK Rate of demand page-ins from DASD for blocked pages.