MapReduce section
The MapReduce section of the application profile contains settings related to the MapReduce application and resources assigned to the MapReduce consumer associated with the application. This section is mandatory for MapReduce applications and must be defined only once in each application profile.
Description: Defines MapReduce settings.
Description: Resource groups from which resources are requested to run service instance managers and services to perform computations for clients. The resource group must contain compute hosts, not management hosts.
- reduceLimitPerHost
- Description: Specifies the maximum number of reduce tasks that can run on a host.
Where used:
Required or optional: Optional
Valid values: 0 to 2147483647. A value of -1 indicates an unlimited number of reduce tasks can be run on a host.
Default value: -1
- runMapTaskOnReduceSlot
- Description: Specifies whether map tasks can run on reduce slots, when the number of available
map slots is less than those required to run map tasks but the number of available reduce slots
exceeds those required to run reduce tasks.
Where used:
Required or optional: Optional
Valid values: true or false
Default value: true
- runReduceTaskOnMapSlot
- Description: Specifies whether reduce tasks can run on map slots, when the number of available
reduce slots is less than those required to run reduce tasks but the number of available map slots
exceeds those required to run map tasks.
Where used:
Required or optional: Optional
Valid values: true or false
Default value: true
- targetMrRatio
- Description: Specifies the target slot ratio for running map tasks and reduce tasks in the
resource group. Based on this map-to-reduce ratio, the SSM (application manager) divides resources
into two pools: the map pool and the reducer pool. When the SSM is allocated multiple slots on a
host, the policy tries to balance the M:R ratio on each host. When the SSM is allocated a single
slot on a host, the policy tries to balance the overall M:R ratio for the policy.
Where used:
Required or optional: Optional
Valid values: X:Y, where both X and Y are positive integers with a maximum of 100 and at least one of them is not zero.
Default value: 1:1
Examples: 0:2, 2:0, or 3:4
Description: Defines the configuration for each resource group.
- name
- Description: Resource group from which resources are requested to run service instance managers
and services to perform computations for clients. The resource group name must be unique within the
cluster and must contain compute hosts, not management hosts. The resource group name cannot contain
multiple resource groups (for example, rg1 rg2 rg3), but you can use the asterisk (*) character to
specify a common configuration for all resource groups associated with the MapReduce consumer
(MapReduceConsumer). Each resource group must be defined only once, with the
hosts in each resource group belonging only to that resource group.
Not required for IBM Spectrum Symphony Developer Edition.
Where used:
Required or optional: Required
Valid values: *, or any alphanumeric string that does not include any special characters:- * indicates all resource groups available to the MapReduce consumer (MapReduceConsumer). For example, name="*".
- Any alphanumeric string that does not include any special characters will override settings specified for resource groups included in *.
- reduceLimitPerHost
- Description: Specifies the maximum number of reduce tasks that can run on a host.
If you specify this setting, this value overrides the value specified in ResourceGroups. If this setting is not specified, the value specified in ResourceGroups will apply. If ResourceGroup name is specified as * (that is,
name="*"), this setting applies to all resource groups associated with the consumer (MapReduceConsumer).Where used:
Required or optional: Optional
Valid values: 0 or more. A value of -1 indicates an unlimited number of reduce tasks can be run on a host.
Default value: -1
- runMapTaskOnReduceSlot
- Description: Specifies whether map tasks can run on reduce slots, when the number of available
map slots is less than those required to run map tasks but the number of available reduce slots
exceeds those required to run reduce tasks.
If you specify this setting, this value overrides the value specified in ResourceGroups. If this setting is not specified, the value specified in ResourceGroups will apply. If ResourceGroup name is specified as * (that is,
name="*"), this setting applies to all resource groups associated with the consumer (MapReduceConsumer).Where used:
Required or optional: Optional
Valid values: true or false
Default value: true
- runReduceTaskOnMapSlot
- Description: Specifies whether reduce tasks can run on map slots, when the number of available
reduce slots is less than those required to run reduce tasks but the number of available map slots
exceeds those required to run map tasks.
If you specify this setting, this value overrides the value specified in ResourceGroups. If this setting is not specified, the value specified in ResourceGroups will apply. If ResourceGroup name is specified as * (that is,
name="*"), this setting applies to all resource groups associated with the consumer (MapReduceConsumer).Where used:
Required or optional: Optional
Valid values: true or false
Default value: true
- targetMrRatio
- Description: Specifies the target slot ratio for running map tasks and reduce tasks in the
specified resource group. Based on this map-to-reduce ratio, the SSM (application manager) divides
resources into two pools: the map pool and the reducer pool. When the SSM is allocated multiple
slots on a host, the policy tries to balance the M:R ratio on each host. When the SSM is allocated a
single slot on a host, the policy tries to balance the overall M:R ratio for the policy.
If you specify this setting, this value overrides the value specified in ResourceGroups. If this setting is not specified, the setting specified in ResourceGroups will apply. If ResourceGroup name is specified as * (that is,
name="*"), this setting applies to all resource groups associated with the consumer (MapReduceConsumer).Where used:
Required or optional: Optional
Valid values: X:Y, where both X and Y are positive integers with a maximum of 100 and at least one of them is not zero.
Default value: 1:1
Examples: 0:2, 2:0, or 3:4