Sample configuration and AT-TLS policy rules for z/OS Spark client authentication

The following examples show the sample configuration settings and AT-TLS policy rules that you can use in your spark-env.sh and spark-defaults.conf (both located in the SPARK_CONF_DIR directory) and TCPIP-TTLS.policy AT-TLS policy file, under each of the z/OS Spark client authentication models. They assume the network port configurations as shown in Table 1, and you should modify them to values that are suitable for your environment.

Network port configurations

Table 1. Example network port configurations
Port name	Default port number	Port number in example configuration
Master port	7077	7077
Mast REST port	6066	6066
Worker port	(random)	7177
Block manager port	(random)	7511
External shuffle server	7337	7337
PySpark daemon	(random)	7722
Driver port	(random)	7277
Driver block manager port	(value of `spark.blockManager.port`)	7611

For more information about configuring the Spark configuration files for z/OS Spark client authentication, see Configuring additional authorities and permissions for the Spark cluster.

For more information about configuring the AT-TLS policies for z/OS Spark client authentication, see Defining the AT-TLS policy rules.

You can find detailed information about the syntax of each AT-TLS policy statements in "AT-TLS policy statements in z/OS Communications Server: IP Configuration Reference

When AT-TLS is the client authentication method.

spark-defaults.conf

# Set this value to false if you want to disable client authentication on the master port
# The default is true.  This option only applies to client deploy mode. 
# spark.zos.master.authenticate.method indicates the authentication method to use.
spark.zos.master.authenticate      true

# Method used for client authentication. Valid values are ATTLS (default) and TrustedPartner.
# Only applicable if spark.zos.master.authenticate is enabled. See IzODA Installation 
# and Customization Guide for more information on required configuration for each method. 
spark.zos.master.authenticate.method    ATTLS

# The REST server does not support client authentication nor application-layer TLS.
# Only enable this once you have adequate security in place for the REST port.
spark.master.rest.enabled          false

spark.driver.blockManager.port 7611
spark.driver.port 7277
spark.blockManager.port 7511
spark.python.daemon.port 7722 

# uncomment and set these if not using default values
# spark.master.rest.port 6066
# spark.shuffle.service.port 7337

# uncomment this to enable external shuffle server
# spark.shuffle.service.enabled    true

spark-env.sh

# uncomment and set this if not using default value
# SPARK_MASTER_PORT=7077
SPARK_WORKER_PORT=7177

AT-TLS policy rules

##
## AT-TLS POLICY AGENT CONFIGURATION FILE FOR SPARK ON Z/OS
##
#################################################################
## SparkClusterGrp_ATTLS contains the PortRange references
## for the Spark cluster ports that support AT-TLS security.
##
## For detailed usage information and configuration 
## instructions of these ports, please refer to the
## IzODA Installation and Customization Guide
##
## When port-binding fails (due to, e.g., port already in
## use), it will retry on sequential ports to a number of times
## equal the value of spark.port.maxRetries (default: 16).
## This value can be configured in spark-defaults.conf.
##
## (The Shuffle Server port does not support port range retry.)
##
## Considering this behavior, we recommend configuring
## a PortRange to account for the retry behavior for
## ports that support port range retry:
##
## port_number - (port_number + spark.port.maxRetries)
#################################################################
PortGroup                         SparkClusterGrp_ATTLS
{
 PortRangeRef                     SparkMaster_ATTLS
 PortRangeRef                     SparkMasterRest_ATTLS
 PortRangeRef                     SparkExtShuffleServer_ATTLS
}
PortRange                        SparkMaster_ATTLS
{
  Port                           7077-7093
}
PortRange                        SparkMasterRest_ATTLS
{
  Port                           6066-6082
}
PortRange                        SparkExtShuffleServer_ATTLS
{
  Port                           7337
}
#################################################################
##
## KeyRing_ATTLS defines the keyring that will be used during
## Spark AT-TLS authentication.
##
#################################################################
TTLSKeyRingParms                KeyRing_ATTLS
{
  Keyring                       SparkRing
}
#################################################################
##
## SparkServer_ATTLS and SparkClient_ATTLS are the rules that work
## together to encrypt the network traffic among the ports defined
## in the SparkClusterGrp_ATTLS section.
##
#################################################################
TTLSRule                          SparkServer_ATTLS
{
  Direction                       Inbound
  LocalPortGroupRef               SparkClusterGrp_ATTLS
  TTLSGroupActionRef              GroupAct_TTLS_On
  TTLSEnvironmentActionRef        EnvAct_SparkServer_ATTLS
}
TTLSRule                          SparkClient_ATTLS
{
  Direction                       Outbound
  RemotePortGroupRef              SparkClusterGrp_ATTLS
  TTLSGroupActionRef              GroupAct_TTLS_On
  TTLSEnvironmentActionRef        EnvAct_SparkClient_ATTLS
}
#################################################################
##
## EnvAct_SparkServer_ATTLS and EnvAct_SparkClient_ATTLS
## establish the environment for the connections that match the 
## corresponding TTLSRules, using the role and keyring specified.
##
#################################################################
TTLSEnvironmentAction             EnvAct_SparkServer_ATTLS
{
  HandshakeRole                   ServerWithClientAuth
  EnvironmentUserInstance         0
  TTLSKeyRingParmsRef             KeyRing_ATTLS
  TTLSEnvironmentAdvancedParmsRef EnvAdv_TLS
}
TTLSEnvironmentAction             EnvAct_SparkClient_ATTLS
{
  HandshakeRole                   Client
  EnvironmentUserInstance         0
  TTLSKeyRingParmsRef             KeyRing_ATTLS
  TTLSEnvironmentAdvancedParmsRef EnvAdv_TLS
}
#################################################################
##
## GroupAct_TTLS_On is the group action that enables TLS
## security for connections utilizing it.
##
#################################################################
TTLSGroupAction                   GroupAct_TTLS_On
{
TTLSEnabled                       On
}
#################################################################
##
## EnvAdv_TLS is an advanced environment parms object
## enforcing the following conditions on the server:
##     - The server can only utilize TLSv1.2 when accepting
##       a connection,
##     - The server must use level 2 client authentication
##       in the TLS handshake.
##
#################################################################
TTLSEnvironmentAdvancedParms      EnvAdv_TLS
{
  ClientAuthType                  SAFCheck 
  TLSv1                           Off
  TLSv1.1                         Off 
  TLSv1.2                         On
}

When Trusted Partner is the client authentication method.

spark-defaults.conf

# Set this value to false if you want to disable client authentication on the master 
# port. The default is true.  This option only applies to client deploy mode. 
# spark.zos.master.authenticate.method indicates the authentication method to use.
spark.zos.master.authenticate      true

# Method used for client authentication. Valid values are ATTLS (default) and TrustedPartner.
# Only applicable if spark.zos.master.authenticate is enabled. See IzODA Installation 
# and Customization Guide for more information on required configuration for each method.
spark.zos.master.authenticate.method  TrustedPartner

# The REST server does not support client authentication nor application-layer TLS.
# Only enable this once you have adequate security in place for the REST port.
spark.master.rest.enabled          false

spark.driver.blockManager.port 7611
spark.driver.port 7277
spark.blockManager.port 7511
spark.python.daemon.port 7722

# uncomment and set these if not using default values
# spark.master.rest.port 6066
# spark.shuffle.service.port 7337

# uncomment this to enable external shuffle server
# spark.shuffle.service.enabled    true

spark-env.sh

" uncomment and set this if not using default value
" SPARK_MASTER_PORT=7077 
SPARK_WORKER_PORT=7177

Trusted Partner policy rules

##
## TRUSTED PARTNER POLICY AGENT CONFIGURATION FILE FOR SPARK ON Z/OS
##
#################################################################
## SparkClusterGrp_TP contains the PortRange references
## for the Spark cluster ports that support Trusted Partner
## security.
##
## For detailed usage information and configuration 
## instructions of these ports, please refer to the
## IzODA Installation and Customization Guide
##
## When port-binding fails (due to, e.g., port already in
## use), it will retry on sequential ports to a number of times
## equal the value of spark.port.maxRetries (default: 16).
## This value can be configured in spark-defaults.conf.
##
## (The Shuffle Server port does not support port range retry.)
##
## Considering this behavior, we recommend configuring
## a PortRange to account for the retry behavior for
## ports that support port range retry:
##
## port_number - (port_number + spark.port.maxRetries)
#################################################################
PortGroup                         SparkClusterGrp_TP
{
  PortRangeRef                    SparkMaster_TP
}
PortRange                         SparkMaster_TP
{
  Port                            7077-7093
}
#################################################################
##
## KeyRing_TP defines the keyring that will be used during
## Spark Trusted Partner authentication.
##
#################################################################
TTLSKeyRingParms                  KeyRing_TP
{
  Keyring                         SparkRingTP
}
#################################################################
##
## SparkServer_TP and SparkClient_TP are the rules that work
## together to encrypt the network traffic among the ports defined
## in the SparkClusterGrp_TP section.
##
#################################################################
TTLSRule                          SparkServer_TP
{
  Direction                       Inbound
  LocalPortGroupRef               SparkClusterGrp_TP
  TTLSGroupActionRef              GroupAct_TTLS_On
  TTLSEnvironmentActionRef        EnvAct_SparkServer_TP
}
TTLSRule                          SparkClient_TP
{
  Direction                       Outbound
  RemotePortGroupRef              SparkClusterGrp_TP
  TTLSGroupActionRef              GroupAct_TTLS_On
  TTLSEnvironmentActionRef        EnvAct_SparkClient_TP
}
#################################################################
##
## EnvAct_SparkServer_TP and EnvAct_SparkClient_TP
## establish the environment for the connections that match the 
## corresponding TTLSRules, using the role and keyring specified.
##
#################################################################
TTLSEnvironmentAction             EnvAct_SparkServer_TP
{
  HandshakeRole                   Server
  EnvironmentUserInstance         0
  TTLSKeyRingParmsRef             KeyRing_TP
  TTLSEnvironmentAdvancedParmsRef EnvAdv_TLS_TP
}
TTLSEnvironmentAction             EnvAct_SparkClient_TP
{
  HandshakeRole                   Client
  EnvironmentUserInstance         0
  TTLSKeyRingParms
  {  
   Keyring                       *AUTH*/*
  }
  TTLSEnvironmentAdvancedParmsRef EnvAdv_TLS_TP
}
#################################################################
##
## GroupAct_TTLS_On is the group action that enables TLS
## security for connections utilizing it.
##
#################################################################
TTLSGroupAction                   GroupAct_TTLS_On
{
  TTLSEnabled                     On
}
#################################################################
##
## EnvAdv_TLS_TP is an advanced environment parms object
## enforcing the following conditions on the server:
##     - The server can only utilize TLSv1.2 when accepting
##       a connection.
##
#################################################################
TTLSEnvironmentAdvancedParms      EnvAdv_TLS_TP
{
  TLSv1                           Off
  TLSv1.1                         Off
  TLSv1.2                         On
}