pmr-env.sh reference
The pmr-env.sh configuration file applies only to MapReduce workload, which is available only with the IBM® Spectrum Symphony Advanced Edition. An entitlement key is required to enable the MapReduce framework.
The pmr-env.sh file adopts a shell-script format similar to Hadoop's hadoop-env.sh and sets up the environment for MapReduce applications in IBM Spectrum Symphony. The settings in this file apply only to the local host.
Important: If you define settings for an application as environment variables in the
Service > osTypes > osType > env section of the application profile, the values
that are defined in the application profile override the corresponding values defined in
pmr-env.sh.
Location
This file is installed with IBM Spectrum Symphony at $SOAM_HOME/mapreduce/conf.
Environment variables
| Environment variable | Description |
|---|---|
| JAVA_HOME | Specifies the directory where the correct JRE (Java™ runtime environment) version is installed on your host. The MapReduce framework in IBM Spectrum Symphony requires Oracle Java 1.6 or later. |
| HADOOP_VERSION | Specifies the version of Hadoop installed in your cluster. For Hadoop use
For Cloudera Distribution including Hadoop (CDH), use cdh5.5.1 = CDH 5.5.1 |
| PMR_EXTERNAL_CONFIG_PATH | Specifies the configuration directory of third-party installations to be added to the Java class path. Use this variable to define the configuration directory to, for example, your Hadoop and HBase installations. |
| JVM_OPTIONS | Specifies options to configure your JVM (Java virtual machine). If you are using IBM JRE, add the -Xshareclasses option to improve performance (see IBM Java documentation for suboptions). |
| PMR_SERVICE_DEBUG_PORT | Specifies the port that is used to debug the map, reduce, and combine functions of the MapReduce service while the service is running in its test or production environment and relates to live service debugging. |
| PMR_MRSS_SHUFFLE_CLIENT_PORT | Specifies the port that is used by the shuffle service (mrss), which is by
default BASEPORT+10. If you use the default base port of 7869, the shuffle service port is 7879.
Note: The shuffle service port is listed in pmr-env.sh and in
$EGO_ESRVDIR/esc/conf/services/mrss.xml. If you change the port, ensure that
you update the port number in both configuration files.
|
| PMR_MRSS_SHUFFLE_DATA_WRITE_PORT | Specifies the port that is used by the shuffle service (mrss) for data writes,
which is by default BASEPORT+11. If you use the default base port of 7869, the shuffle service port
for data writes is 7880. Note: The shuffle service port is listed in pmr-env.sh
and in $EGO_ESRVDIR/esc/conf/services/mrss.xml. If you change the port, ensure
that you update the port number in both configuration files.
|
| PYTHON_PATH | Specifies the directory where Python is installed on your host. |
| PATH | Specifies the directories to executables on your host. |
| USER_CLASSPATH | Specifies the directory to user Java class files, JAR files, or both. |
| JAVA_LIBRARY_PATH | Specifies the directory from where the MapReduce client and service (MRService) load native libraries. |
| CLOUDERA_HOME | Specifies the directory where Cloudera's Distribution including Hadoop (CDH) is installed. |
Example
#!/usr/bin/env bash
#Set IBM Platform MapReduce environment variables here.
# The Java implementation to use
export JAVA_HOME=/opt/java/jre1.6.0_25_64bits/
# The version of Hadoop
export HADOOP_VERSION=1_0_0
# The configuration file path which will be added to the Java classpath; e.g. $HADOOP_HOME/conf
export PMR_EXTERNAL_CONFIG_PATH=/exampleuser/hadoop-1.0.0/conf
# The JVM options. If using IBM JRE, add '-Xshareclasses' to improve performance (see IBM Java documentation for details).
export JVM_OPTIONS=-Xmx512m
# The port used to debug the map, reduce, and combine functions of the MapReduce service while it is running in its test or production environment
export PMR_SERVICE_DEBUG_PORT=
# The port used by the shuffle service (mrss). If you change the value here, ensure that you also change the value in the $EGO_ESRVDIR/esc/conf/services/mrss.xml file.
export PMR_MRSS_SHUFFLE_CLIENT_PORT=35010
# The port used by the shuffle service (mrss) for data writes
export PMR_MRSS_SHUFFLE_DATA_WRITE_PORT=35012
# The path to your Python installation
export PYTHON_PATH=/bin:/usr/bin:/usr/local/bin
# The path to executable programs
export PATH=${PATH}:${JAVA_HOME}/bin:${PYTHON_PATH}
# The user classpath which will be added to the Java classpath
export USER_CLASSPATH=
#The path where the Platform MapReduce client and the MapReduce service load native libraries
export JAVA_LIBRARY_PATH=/exampleuser/hadoop-1.0.0/lib/native/Linux-amd64-64/:/exampleuser/hadoop-1.0.0/lib/native/Linux-i386-32/
#The path to your Cloudera MapReduce installation
export CLOUDERA_HOME=/exampleuser/hadoop-1.0.0/