Defining WLM classification rules for Spark

After you have the service classes and, optionally, the report classes defined for Spark, you can define classification rules for your Spark work. WLM uses classification rules to map work coming into the system to a specific service class and report class. The classification is based on work qualifiers. The first qualifier is the subsystem type that receives the work request. The subsystem type for work that is processed in z/OS® UNIX (which Spark workloads are) is OMVS.

The OMVS subsystem type supports the following secondary qualifiers:

Sysplex name (PX)
System name (SY)
Transaction name or job name (TN)
User ID (UI)
Accounting Information (AI)

For instance, if you use a dedicated user ID to start the Spark cluster, this user ID could serve as a classification mechanism. If you have enabled client authentication, you can also define a separate service class for each user based on the user ID and job name. For more information about enabling client authentication, see Configuring z/OS Spark client authentication.

You can also use masking (%) or wildcard (*) notation as a way to group work to the same service class or report class. In earlier examples, all Spark job names are prefixed with ODAS. You can easily group these jobs into the same WLM service class by specifying a classification rule for the OMVS subsystem by using a qualifier of transaction name ODAS*. Alternatively, you can place the master, worker, and driver in one service class (based on transaction name qualifiers of ODASM*, ODASW*, and ODASD*, respectively) and place the executors in another service class (based on a transaction name qualifier of ODASX*).

Tip: IBM® suggests classifying any significant Spark workload into its own service class or classes.

Note: z/OS IzODA Mainframe Data Service (FMID HMDS120) uses different classification rules. For more information about configuring WLM for the Data Service server, see Customizing the Data Service server.

To create classification rules, select the 6. Classification Rules option on the Definition Menu in the WLM ISPF application.

Example: The following figure shows an example of creating a classification rule that uses the transaction name qualifier ODAS* to group work into the ODASSC1 service class and ODASRC report class:

                  Modify Rules for the Subsystem Type       Row 1 to 1 of 1  
 Command ===> ___________________________________________ Scroll ===> PAGE   
                                                                             
 Subsystem Type . : OMVS        Fold qualifier names?   Y  (Y or N)          
 Description  . . . Sample OMVS classification rule                                 
                                                                             
 Action codes:   A=After     C=Copy        M=Move     I=Insert rule          
                 B=Before    D=Delete row  R=Repeat   IS=Insert Sub-rule     
                                                              More ===>      
          -------Qualifier--------                 -------Class--------      
 Action   Type      Name     Start                  Service     Report       
                                          DEFAULTS: OMVSDFSC    OMVSDFRC     
  ____  1 TN        ODAS*    ___                    ODASSC1     ODASRC

For more information about planning for and using WLM, see z/OS MVS Planning: Workload Management and z/OS MVS Programming: Workload Management Services.