Invoke timeouts for map and reduce tasks

You can configure timeout durations for map and reduce tasks separately, so that the MapReduce framework can kill a task when it does not report progress or indicate that it is alive.

Configure timeouts for map tasks and reduce tasks from the command line during job submission or from a configuration file:
  • pmr.map.invoke.timeout.duration: Specifies the number of seconds the system waits for the invoke method to complete for a map task before a timeout is considered. If the map task does not read input, write output, or update its status string within the configured duration, the map task is killed.
  • pmr.reduce.invoke.timeout.duration: Specifies the number of seconds the system waits for the invoke method to complete for a reduce tasks before a timeout is considered. If the reduce task does not read input, write output, or update its status string within the configured duration, the reduce task is killed.

Specify the timeout duration as any value between "0" and "4294967295". If you specify "0", the task never times out. If you specify a negative integer or a decimal, the system ignores the value and uses the value set for the invoke method in the Service section of the application profile (Service > Control > Method invoke > timeout > duration). When applications take an extended period of time to process, you may want to set a higher timeout, so that the system does not kill the task assuming that it has timed out.

Configuring task timeout using the mrsh utility

To configure the timeout for map and reduce tasks using the command line, add the following options to your job submission command:
  • For map tasks, add -Dpmr.map.invoke.timeout.duration=value
  • For reduce tasks, add -Dpmr.reduce.invoke.timeout.duration=value

where value identifies an integer between 0 and 4294967295. For example:$ mrsh jar jarfile [classname] -Dpmr.map.invoke.timeout.duration=30 -Dpmr.reduce.invoke.timeout.duration=45 [args]

Configuring task timeout in a configuration file

Note: Options set in the mrsh command-line override options set in the configuration file.
  1. Open the pmr-site.xml configuration file at $PMR_HOME/conf.
  2. Add the following properties. For example:
    <property>
  <name>pmr.map.invoke.timeout.duration</name>
  <value>30</value>
</property>

    <property>
  <name>pmr.reduce.invoke.timeout.duration</name>
  <value>45</value>
</property>
  3. Save the file.