Custom configuration properties and environment variables for Spark

Applies to :

Spark engine

Apache Gluten accelerated Spark engine

Custom Configuration Parameters (Spark Configurations)

Properties set using spark.conf.set() or --conf flag that control Spark application behavior. These are application-specific, set at runtime.

Table 1. Default Spark configuration parameters
Spark configuration Default value
ae.spark.driver.log.level Possible values:
  • ALL
  • TRACE
  • DEBUG
  • INFO
  • WARN
  • ERROR
  • FATAL
  • OFF
Default value: ERROR

For information about how to configure the parameter, see Configuring Spark log level information.

ae.spark.executor.log.level

Change the log level of executor. Possible values:

DEBUG, INFO, WARN, ERROR, OFF

For information about how to configure the parameter, see Configuring Spark log level information.

ae.kubernetes.spec.topologySpreadConstraints Base64 encoded value of topologySpreadConstraints supported by k8s.

For information about how to configure the parameter, see Setting custom configurations.

ae.kubernetes.spec.tolerations Base64 encoded value of tolerations supported by k8s.
ae.kubernetes.spec.nodeSelector Provide list of labels to identify the nodes to schedule pods ON.
ae.spark.application.priority To set the priority of the Spark applications. Can be used only when ibm cpd schedular is enabled. Higher the value, higher the priority.
ae.spark.log.dir Set the log directory for the Spark applications.
spark.driver.resource.gpu.amount Number of GPUs required for the Spark driver.

For information about how to configure the parameter, see Enabling NVIDIA GPU acceleration.

spark.driver.resource.gpu.vendor Set the value to nvidia to indicate GPU vendor.

For information about how to configure the parameter, see Enabling NVIDIA GPU acceleration.

spark.executor.resource.gpu.amount Number of GPUs required for each Spark executor.

For information about how to configure the parameter, see Enabling NVIDIA GPU acceleration.

spark.executor.resource.gpu.vendor Set the value to nvidia.com.

For information about how to configure the parameter, see Enabling NVIDIA GPU acceleration.

spark.eventLog.enabled TRUE
spark.executor.extraClassPath /home/spark/space/assets/data_asset/*:/home/spark/user_home/dbdrivers/*:/cc-home/_global_/dbdrivers/*:/home/spark/shared/user-libs/spark2/*:/home/spark/user_home/dbdrivers/*:/home/spark/shared/user-libs/common/*:/home/spark/shared/user-libs/connectors/*:/opt/ibm/connectors/parquet-encryption/*:/opt/ibm/third-party/libs/spark2/*:/opt/ibm/third-party/libs/common/*:/opt/ibm/third-party/libs/connectors/*:/opt/ibm/spark/external-jars/*
spark.executer.memory 1 G
spark.executer.cores 1
(custom) ae.spark.executor.count 1
(custom) ae.spark.application.priority 1
spark.driver.extraClassPath /home/spark/space/assets/data_asset/*:/home/spark/user_home/dbdrivers/*:/cc-home/_global_/dbdrivers/*:/home/spark/shared/user-libs/spark2/*:/home/spark/user_home/dbdrivers/*:/home/spark/shared/user-libs/common/*:/home/spark/shared/user-libs/connectors/*:/opt/ibm/connectors/parquet-encryption/*:/opt/ibm/third-party/libs/spark2/*:/opt/ibm/third-party/libs/common/*:/opt/ibm/third-party/libs/connectors/*:/opt/ibm/spark/external-jars/*
spark.driver.memory 1024 M
spark.driver.cores 1
spark.local.dir /tmp/spark/scratch See spark.local.dir configuration parameter for details.
spark.master.ui.port 8080
spark.worker.ui.port 8081
spark.ui.port 4040
spark.history.ui.port 18080
spark.ui.enabled TRUE
spark.ui.killEnabled FALSE
spark.eventLog.dir file:///home/spark/spark-events
spark.ui.reverseProxy TRUE
spark.ui.showConsoleProgress TRUE
spark.shuffle.service.port 7337
spark.r.command /opt/ibm/conda/R/bin/Rscript
spark.hadoop.fs.s3a.fast.upload TRUE
spark.hadoop.fs.s3a.multipart.size 33554432
spark.hadoop.fs.stocator.scheme.list cos
spark.hadoop.fs.stocator.cos.scheme cos
spark.hadoop.fs.stocator.glob.bracket.support TRUE
spark.hadoop.fs.stocator.cos.impl com.ibm.stocator.fs.cos.COSAPIClient
spark.hadoop.fs.cos.impl com.ibm.stocator.fs.ObjectStoreFileSystem
spark.hadoop.fs.s3a.impl org.apache.hadoop.fs.s3a.S3AFileSystem
spark.authenticate FALSE
spark.network.crypto.enabled FALSE
spark.network.crypto.keyLength 256
ae.spark.autoscale.enable Signals that the application will autoscale based on the application's demand and any other autoscaling configurations that are set. If specified at instance level, all applications in the instance will autoscale. Default value:false.

For information about how to configure the parameter, see Autoscaling.

spark.dynamicAllocation.initialExecutors Specifies the initial number of executors to be created, irrespective of any demand made by the Spark application. Default value:0
spark.dynamicAllocation.minExecutors Specifies the minimum number of executors to be maintained, irrespective of any demand by the Spark application. Default value:0
spark.dynamicAllocation.maxExecutors Specifies the maximum number of executors to be created irrespective of any demand by the Spark Application. Default value:2
ae.spark.autoscale.scaleupDemandPercentage Specifies the percentage of the executors demanded by Spark Application that should be fulfilled by the application auto-scaler. For example, if at a certain stage a Spark application requests 10 executors and the value for this configuration is set to 50 percent, the application auto-scaler will scale up the number of executors by 5 executors. The default value of this configuration is 100 percent which essentially means that any number of executors requested by an application's demand can be added by the application auto-scaler. Default value:100
ae.spark.autoscale.scaledownExcessPercentage Specifies the percentage of the idle executors held up by the Spark application that are to be removed by the application auto-scaler. For example, if after completing a certain stage in a Spark application, there are 20 idle executors held up by the application and the value for this configuration is set to 25 percent, the application auto-scaler will scale down the executors in the application by 4 executors. The default value for this configuration is 100 percent which means that the application auto-scaler can scale down all idle executors. Default value:100
ae.spark.autoscale.frequency The frequency in seconds at which the application auto-scaler scales up executors or scales down executors in an application. Default value:10s
spark.sql.extensions To enable the Spark access control extension, you must set the Spark configuration to authz.IBMSparkACExtension. For information about how to configure the parameter, see Enforcing data access policies for Spark runtime using Spark access control extension.
idempotency_key Use the idempotency_key parameter in the Spark application payload to avoid triggering duplicate job submissions. This ensures that repeated requests with the same key are treated as a single request. Idempotency_key cannot be greater than 64 characters in length. For information about how to configure the parameter, see Configuring Idempotency key in payload.
timeout_in_seconds Use the timeout_in_seconds parameter in the payload to specify the maximum execution time while submitting an application. For information about how to configure the parameter, see Configuring maximum runtime time in payload.
max_retries Use the max_retries parameter in the payload to specify the maximum execution time while submitting an application. For information about how to configure the parameter, see Configuring maximum retries in payload.
min_retry_interval_in_seconds Use min_retry_interval_in_seconds field in the payload to specify Retry feature. For information about how to configure the parameter, see Configuring minimum retries in payload.
ae.spark.kubernetes.container.imageae.spark.kubernetes.container.registry.nameae.spark.kubernetes.container.registry.usernameae.spark.kubernetes.container.registry.password Use these configurations to add system packages, Python libraries, or JAR files required by your workloads. For information about how to configure the parameter, see Submitting a Spark application by using custom runtime image.
init_scripts Initialization scripts are shell scripts that run on every cluster node at startup before the Spark driver or executor launch. They are especially useful when your Spark application requires custom packages, environment variables, or perform any one‑time spark runtime configuration. For information about how to configure the parameter, see Seamless package management using init scripts.
spark.executor.logs.rolling.strategyspark.executor.logs.rolling.time.intervalspark.executor.logs.rolling.maxRetainedFiles Configure Spark log rotation based on time, and old logs are cleaned up automatically. For information about how to configure the parameter, see Enable Spark log and event log rotation.
spark.eventLog.rolling.enabledspark.eventLog.rolling.maxFileSize Configure Spark log rotation based on size, and old logs are cleaned up automatically. With spark.eventLog.rolling.enabled=true and maxFileSize configured, event logs are correctly split into multiple files (events_1, events_2, …). For information about how to configure the parameter, see Enable Spark log and event log rotation.
spark.history.fs.cleaner.enabledspark.history.fs.cleaner.intervalspark.history.fs.cleaner.maxAge Configure history server log clean up automatically. For information about how to configure the parameter, see Enable Spark log and event log rotation.

Environment variables

System-level variables that configure Spark's runtime environment and JVM settings. The cluster-wide configurations set before Spark starts.

The following table lists the environment variables and their defaults:

Table 2. Default Spark environment variables
Environment variable Default value
SPARK_DIST_CLASSPATH /home/spark/space/assets/data_asset/*:/home/spark/user_home/dbdrivers/*:/cc-home/_global_/dbdrivers/*:/opt/ibm/connectors/idax/*:/opt/ibm/connectors/cloudant/*:/opt/ibm/connectors/db2/*:/opt/ibm/connectors/others-db-drivers/*:/opt/ibm/connectors/wdp-connector-driver/*:/opt/ibm/connectors/wdp-connector-jdbc-library/*:/opt/ibm/connectors/stocator/*:/opt/ibm/connectors/s3/*:/opt/ibm/image-libs/common/*:/opt/ibm/image-libs/spark2/*:/opt/ibm/third-party/libs/batch/*:/opt/ibm/spark/external-jars/*
SPARK_LOCAL_DIRS /tmp/spark/scratch
SPARK_MASTER_WEBUI_PORT 8080
SPARK_MASTER_PORT 7077
SPARK_WORKER_WEBUI_PORT 8081
CLASSPATH /home/spark/user_home/dbdrivers/*:/opt/ibm/connectors/idax/*:/opt/ibm/connectors/cloudant/*:/opt/ibm/connectors/db2/*:/opt/ibm/connectors/others-db-drivers/*:/opt/ibm/connectors/wdp-connector-driver/*:/opt/ibm/connectors/wdp-connector-jdbc-library/*:/opt/ibm/connectors/stocator/*:/opt/ibm/connectors/s3/*:/opt/ibm/image-libs/common/*:/opt/ibm/image-libs/spark2/*:/opt/ibm/third-party/libs/batch/*
LD_LIBRARY_PATH /opt/ibm/connectors/dsdriver/dsdriver/lib:/opt/ibm/connectors/others-db-drivers/oracle/lib:/opt/ibm/jdk/jre/lib/architecture/server:/opt/ibm/jdk/jre/lib/architecture/:/usr/local/lib:/lib64
RUNTIME_PYTHON_ENV python310
PYTHONPATH /home/spark/space/assets/data_asset:/home/spark/user_home/python-3:/cc-home/_global_/python-3:/home/spark/shared/user-libs/python:/home/spark/shared/conda/envs/python/lib/python/site-packages:/opt/ibm/conda/miniconda/lib/python/site-packages:/opt/ibm/third-party/libs/python3:/opt/ibm/image-libs/python3:/opt/ibm/image-libs/spark2/xskipper-core.jar:/opt/ibm/image-libs/spark2/spark-extensions.jar:/opt/ibm/image-libs/spark2/metaindexmanager.jar:/opt/ibm/image-libs/spark2/stmetaindexplugin.jar:/opt/ibm/spark/python:/opt/ibm/spark/python/lib/py4j-0.10.7-src.zip
R_LIBS_USER /home/spark/space/assets/data_asset:/home/spark/shared/user-libs/R:/opt/ibm/third-party/libs/R:/opt/ibm/conda/R/lib64/R/library/:/opt/ibm/spark/R/lib:/opt/ibm/image-libs/R