mds-scheduler.xml reference

The mds-scheduler.xml configuration file applies to a EGO-YARN environment, and defines how resource manager queues interact. Within the EGO-YARN environment, the multidimensional scheduler in IBM® Spectrum Symphony provides multitenant sharing policies.

Prerequisites

EGO-YARN is available only with the IBM Spectrum Symphony Advanced Edition on Linux®. An entitlement key is required to enable integration with your existing YARN environment.
For the supported YARN versions, see Supported distributed files systems for MapReduce or YARN integration.
To use EGO-YARN, you must have a supported version of YARN installed and configured on your management host before installing IBM Spectrum Symphony.

Location

This configuration file is copied to the Hadoop configuration directory, which by default is $HADOOP_YARN_HOME/etc/hadoop/mds-scheduler.xml.

Example

In the following example mds-scheduler.xml file, note the following:

<AppMasterConf>: Specifies configuration for the application server, which is the first task in a job and is responsible for requesting resources for other tasks in the job. In open-source YARN, the application sever can be started on any node in the YARN cluster. In EGO-YARN, it can be configured in a dedicated resource group, which can be bound to a group of more reliable hosts.
<prestartnms>true</prestartnms>: Indicates that reserved resources configured for multidimensional scheduling can have prestarted node managers on them.

Here is an example file:

<allocations>
	<resourceMontiorInterval>15</resourceMontiorInterval> 
	<AppMasterConf> 
		<consumer>YARNAMConsumer</consumer>
		<resourcePlan>YARNAMDistributeTree</resourcePlan>
		<prestartnms>true</prestartnms>
	</AppMasterConf>
	<queue name="default">
		<consumer>YARNComputeConsumer</consumer> 
		<resourcePlan>YARNComputeHostsDistributeTree</resourcePlan> 
		<prestartnms>true</prestartnms> 
		<maxRunningApps>50</maxRunningApps> 
		
	</queue>
	<queue name="sampleQueue">
		<consumer>YARNComputeConsumer</consumer>
		<resourcePlan>YARNComputeHostsDistributeTree</resourcePlan>
		<prestartnms>true</prestartnms>
		<maxRunningApps>50</maxRunningApps>
	</queue>
</allocations>