Measuring API workloads with WLM

Using WLM to classify API requests

WLM uses classification rules to associate incoming requests with service classes that specify appropriate performance goals. There are two types of WLM classification to manage z/OS Connect.

STC type classification

z/OS Connect Servers can be classified as STCs (started tasks) in order to manage all of the server activity that does not relate to specific transactions, for example, garbage collector activity and connection pool management.

Note: An z/OS Connect Server can also be started as a submitted job, in which case you would use an equivalent JES classification.

CB type classification

Every request to an z/OS Connect Server is dispatched on a USS (UNIX System Services) thread and executes under a WLM enclave that is classified as a CB workload transaction.

The processor activity of z/OS Connect transactions is reported to WLM and RMF and so can be measured in RMF Workload Activity reports for service classes and report classes. RMF Monitor I provides long-term data collection for system workload and system resource utilization. It measures various areas of system activity over a long time, and writes the data into SMF record types 70 and 72. The RMF Post Processor is used to create reports from RMF Monitor I data

With the recent WLM enhancements, you can now also tag z/OS Connect requests with MOBILE reporting attribute, see article Limiting the impact of API workloads on MLC.

A transaction class provides a granular approach for classifying API requests. For instance, you can classify requests for different API operations inside the same z/OS Connect Server depending on the request URI.

A transaction class for z/OS Connect is enabled by using the zosWlm-1.0 feature. After adding this feature, validate that the z/OS Connect Server is authorized to use the feature by checking for the message CWWKB0103I: Authorized service group ZOSWLM is available in the messages log. For more information on authorizing z/OS Connect to use WLM, refer to Enabling workload management for Liberty on z/OS.

Using a transaction class to classify API requests

Transaction classes are defined in the server.xml file. Figure 1 shows an example classification of 3 API operations for the sample catalog API. The catalog API is used to access the CICS® Catalog Manager sample application. For more information on the catalog API, refer to the z/OS Connect Getting Started Guide.

<zosWorkloadManager collectionName="MOPZCET"/>
<wlmClassification>
  <httpClassification
          transactionClass="TCIC"
          method="GET"
          resource="/catalogManager/items"/>
  <httpClassification
          transactionClass="TCIS"
          method="GET"
          resource="/catalogManager/items/*"/>
  <httpClassification
          transactionClass="TCPO"
          method="POST"
          resource="/catalogManager/orders"/>
</wlmClassification>

In Figure 1 we assign transaction class TCIC to the inquire catalog API operation that is invoked by using an HTTP GET request with URI /catalogManager/items. We also assign transaction classes to the inquire single and place order API operations.

The resource attribute of the httpClassification element can be specified by using wildcards. One asterisk represents any number of characters, and is used as a wildcard for the string between two path separator (/) characters. Two consecutive asterisks represent any number of characters that include the path separator. Double asterisks cannot appear with other characters within a pair of path separators.

In the final case, the message CWWKB0164W: WLM CLASSIFication Configuration value for attribute resource uses incorrect wildcarding. is reported in the messages.log file.

For more information about defining a transaction class refer to WLM classification for z/OS in the WebSphere Application Server Liberty documentation.

A collectionName can be provided with the zosWorkloadManager XML element. If not defined, the default value of the collectionName is the server name, the name that appears in the servers directory in UNIX System Services. The collectionName of the z/OS Connect Server is used for WLM classification, so it needs to match the CN name in WLM as shown in Figure 2. This implies that the collectionName value is limited to 8 characters and must be in uppercase. If the server name does not comply with these two conditions, a custom collectionName – complying with the restrictions – must be set with zosWorkloadManager for the API workload to be classified properly.

Figure 2 shows the corresponding WLM classification rules for classifying the catalog API requests.

 Subsystem Type . : CB          Fold qualifier names?   Y  (Y or N)      
 Description  . . . WebSphere requests
        
 Action codes:   A=After     C=Copy        M=Move     I=Insert rule
                 B=Before    D=Delete row  R=Repeat   IS=Insert Sub-rule
                                                              More ===> 
          -------Qualifier--------                 -------Class--------  
 Action   Type      Name     Start                  Service     Report
                                          DEFAULTS: CBENCLAV    RCBDEF   
  _____ 1 CN        MOPZCET    ___                  CBHI        RCBZCET 
  _____ 2   TC        TCIC       ___                CBHI        RCBTCIC 
  _____ 2   TC        TCIS       ___                CBHI        RCBTCIS 
  _____ 2   TC        TCPO       ___                CBHI        RCBTCPO        

Figure 2 defines specific report classes to monitor the different API operations, for example, transaction class TCIC for inquire catalog requests is assigned to the RCBTCIC report class. We will use this report class to report on the number of inquire catalog API operations, the response time, and CPU consumed.

Classifying the z/OS Connect Server

Figure 3 shows the WLM classification rule for classifying the z/OS Connect Server.

 Subsystem Type . : STC         Fold qualifier names?   Y  (Y or N)      
 Description  . . . Started Tasks classifications
        
 Action codes:   A=After     C=Copy        M=Move     I=Insert rule
                 B=Before    D=Delete row  R=Repeat   IS=Insert Sub-rule
                                                              More ===> 
          -------Qualifier--------                 -------Class--------  
 Action   Type      Name     Start                  Service     Report
                                          DEFAULTS: OPSDEF      RSTCDEF
  ____ 1  TN       MOPZCET   ___                    SYSSTC      RSTCZCET 

Figure 3 defines a specific report class RSTCZCET to monitor the z/OS Connect started task (STC) MOPZCET. We will use this report class to report on the server activity that does not get attributed to specific API requests.

Measuring the API

Figure 4 shows an extract from the RMF Workload Activity report for inquire catalog API requests (report class RCBTCIC).

 REPORT BY: POLICY=WLMPOL  REPORT CLASS=RCBTCIC
                           DESCRIPTION =CB RC for inquire catalog API 
 -TRANSACTIONS--  TRANS-TIME HHH.MM.SS.TTT   SERVICE  TIME        ---APPL %---   
 AVG        0.04  ACTUAL                 2   CPU     7.176        CP      1.20 
 MPL        0.04  EXECUTION              1   SRB     0.000        AAPCP   0.00
 ENDED     11900  QUEUED                 0   RCT     0.000        IIPCP   1.20 
 END/S     19.83  R/S AFFIN              0   IIT     0.000                
 £SWAPS        0  INELIGIBLE             0   HST     0.000        AAP      N/A    
 EXCTD         0  CONVERSION             0   AAP       N/A        IIP      N/A 
 AVG ENC    0.04  STD DEV                0   IIP       N/A

 TRANSACTION APPL% :   TOTAL  :  CP  1.20   AAP/IIP ON CP 1.20   AAP/IIP 
                       MOBILE :  CP  0.00   AAP/IIP ON CP 0.00   AAP/IIP  

Figure 4 provides us with the following useful reporting information:

• An average of 19.83 inquire catalog API requests were executed per second during the recording interval (10 minutes).
• A total of 7.176 seconds of CPU time was consumed by the inquire catalog API requests, which equate to 0.6 ms per request.
• The total CPU consumed is equivalent to 1.20% of a CP.
• 100% of the CPU consumed is zIIP eligible.
  Note: The IIPCP value represents the percentage of a CP that might have run on a zIIP if a zIIP had been available on the system.
• No mobile CPU is reported since we have not classified the inquire catalog API requests as being initiated by a mobile device.

Figure 5 shows a similar extract from the RMF Workload Activity report for inquire single API requests (report class RCBTCIS).

 REPORT BY: POLICY=WLMPOL  REPORT CLASS=RCBTCIS
                           DESCRIPTION =CB RC for inquire single API  
 -TRANSACTIONS--  TRANS-TIME HHH.MM.SS.TTT   SERVICE  TIME        ---APPL %---   
 AVG        0.03  ACTUAL                 1   CPU     3.930        CP     0.65        
 MPL        0.03  EXECUTION              1   SRB     0.000        AAPCP   0.00 
 ENDED     11900  QUEUED                 0   RCT     0.000        IIPCP   0.65        
 END/S     19.83  R/S AFFIN              0   IIT     0.000        
 £SWAPS        0  INELIGIBLE             0   HST     0.000        AAP      N/A 
 EXCTD         0  CONVERSION             0   AAP       N/A        IIP      N/A 
 AVG ENC    0.03  STD DEV                0   IIP       N/A

 TRANSACTION APPL% :   TOTAL  :  CP   0.65   AAP/IIP ON CP   0.65   AAP/IIP N/A                      
                       MOBILE :  CP   0.00   AAP/IIP ON CP   0.00   AAP/IIP N/A  

Figure 5 shows that a total of 3.930 seconds of CPU time was consumed by the inquire single API requests, which equate to 0.33 ms per request. Note that this is less than the average CPU cost of inquire catalog API requests because an inquire single request retrieves a single item from the catalog, whereas the inquire catalog request retrieves 15 items.

Reporting on the z/OS Connect Server

Figure 6 shows an extract of an RMF Workload Activity report for report class RSTCZCET. This is the report class that we defined in z/OS Connect Server.

 REPORT BY: POLICY=WLMPOL  REPORT CLASS=RSTCZCET
                           DESCRIPTION =STC classification for MOPZCET
 -TRANSACTIONS--  TRANS-TIME HHH.MM.SS.TTT   SERVICE  TIME        ---APPL %---   
 AVG        1.00  ACTUAL                 0   CPU     5.158       CP       0.92
 MPL        1.00  EXECUTION              0   SRB     0.339       AAPCP    0.00 
 ENDED         0  QUEUED                 0   RCT     0.000       IIPCP    0.85
 END/S      0.00  R/S AFFIN              0   IIT     0.000
 £SWAPS        0  INELIGIBLE             0   HST     0.000       AAP       N/A
 EXCTD         0  CONVERSION             0   AAP       N/A       IIP       N/A
 AVG ENC    0.00  STD DEV                0   IIP       N/A   
 
 TRANSACTION APPL% :   TOTAL  :  CP   0.92   AAP/IIP ON CP   0.85   AAP/IIP  N/A
                       MOBILE :  CP   0.00   AAP/IIP ON CP   0.00   AAP/IIP  N/A 

Figure 6 shows that a total of 5.158 seconds of CPU time was consumed by the server during the reporting interval (10 minutes). The total CPU consumed by the server is equivalent to 0.92% of a CP, out of which 0.85% of the CP is zIIP eligible.

Measuring the CICS transaction

Each request to the catalog API results in the execution of a CICS transaction. We also use WLM to classify the CICS transaction and measure the transaction response time and CPU consumption.

Figure 7 shows the WLM classification rules for classifying the CICS Catalog Manager transactions.

 Subsystem Type . : CB          Fold qualifier names?   Y  (Y or N)
 Subsystem Type . : CICS        Fold qualifier names?   Y  (Y or
        N)      
 Description  . . . CICS Environment
 Action codes:   A=After     C=Copy        M=Move     I=Insert rule
                 B=Before    D=Delete row  R=Repeat   IS=Insert Sub-rule
                                                              More ===> 
          -------Qualifier--------                 -------Class--------  
 Action   Type      Name     Start                  Service     Report
                                          DEFAULTS: CBENCLAV    RCBDEF  
 _____  1 SI        CICSMOBT ___                    CICSHI      RCICMOBT
 _____  2   TN        MZIC     ___                  CICSHI      RCICMZIC
 _____  2   TN        MZIS     ___                  CICSHI      RCICMZIS 
 _____  2   TN        MZPO     ___                  CICSHI      RCICMZPO
    

Figure 7 defines specific report classes to monitor the different CICS transactions, for example, inquire catalog requests invoke CICS transaction MZIC, which is classified in the RCICMZIC report class. We use this report class to report on the number of MZIC transactions, the response time, and CPU consumed.

 REPORT BY: POLICY=WLMPOL         REPORT CLASS=RCICMZIC
                                  DESCRIPTION =RC for CICS transaction MZIC
 -TRANSACTIONS--  TRANS-TIME HHH.MM.SS.TTT   SERVICE  TIME        ---APPL %---   
 AVG        0.00  ACTUAL                 0
 MPL        0.00  EXECUTION              0
 ENDED     11900  QUEUED                 0
 END/S     19.83  R/S AFFIN              0

 TRANSACTION APPL% :   TOTAL  :  CP   0.39   AAP/IIP ON CP   0.00   AAP/IIP  N/A
                       MOBILE :  CP   0.00   AAP/IIP ON CP   0.00   AAP/IIP N/A     

Calculating the cost of an API requester call

SMF72 records capture the CPU usage for all requests within a server. These records can be used to understand the cost of an API requester call.

<featureManager>
    <feature>zosconnect:apiRequester-1.0</feature>
    <feature>zosconnect:zosConnectCommands-1.0</feature>
    <feature>zosWLM-1.0</feature>
</featureManager>

<zosWorkloadManager collectionName="Liberty"/>

<wlmClassification>
    <httpClassification transactionClass="TCAPIREQ" method="PUT" resource="/zosConnect/apiRequesters/*"/>
</wlmClassification>

    Qualifier   Qualifier       Starting     Service   Report
 #  type        name            position     Class     Class
 -  ---------   --------------  --------     -------   -------
 1  CN          LIBERTY
 2  . TC        . TCAPIREQ                             APIREQ

Define a WLM report class APIREQ for API requester.

Summary

This article gives an example of how WLM and RMF can be used to measure the performance of z/OS Connect APIs. Many factors can influence the performance of APIs, including message lengths, security, and back end connectivity.

More information

More information on using WLM to measure z/OS workloads can be found in the ITSO Redbooks publication Using IBM z/OS WLM to Measure Mobile and Other Workloads (REDP-5359-00).

More information on z/OS Connect performance can be found in the IBM z/OS Connect Enterprise Edition Performance Reports.