Consumer section

The Consumer section of the application profile contains settings related to the application and resources assigned to the consumer associated with the application. The applicationName, consumerID, and workloadType elements are mandatory.

Consumer

Description: Defines application and resources assigned to the consumer associated with the application.

applicationName

Description: Name of the application associated with the consumer. The application name must be unique within the cluster. Enclose the application name in double quotation marks if it contains spaces. For example:

name="Commodity Derivatives App"

Where used: Consumer

Required or optional: Required

Valid values: In the cluster management console, characters of any case, 0 to 9, and spaces are allowed. Do not use all as an application name, as that is a reserved word.

autoUnblockPromotedServiceBlockHost

Description: Specifies whether to automatically unblock a host at the application level when new workload arrives for a service that did not block the host. If set to true, this automatic unblocking feature is enabled. If set to false, you must manually unblock the host at the application level to get it back. This feature does not consider whether an administrative block occurred on that host or not.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

collectTaskMetricsInterval

Description: When task-level metrics collection is enabled (with enableTaskBulletin), specifies the interval (in seconds) that task-level metrics are collected. This parameter can only be used on Linux® hosts; if configured on Windows hosts, IBM® Spectrum Symphony ignores the configuration.

IBM Spectrum Symphony collects memory usage data at the interval specified by collectTaskMetricsInterval and only keeps peak memory usage data. A low value increases the accuracy of the memory usage data, but marginally minimalizes performance.

Where used: Consumer

Required or optional: Optional

Valid values: 1 or more up to a maximum of 2147483647

Default value: 5

Related attributes:

enableTaskBulletin: Enables or disables collection of task-level metrics for CPU time and memory usage for reporting purposes.

consumerId

Description: Name and path of the consumer with which to associate this application. The consumer must already exist in IBM Spectrum Symphony. The consumer you associate with the application determines how IBM Spectrum Symphony allocates resources to the application. The consumer ID includes the path to the consumer within the consumer tree.

The consumer must be a leaf consumer. A leaf consumer always resides at the lowest level of the consumer tree.

A unique consumer ID must be specified in IBM Spectrum Symphony, but it is not required to be predefined in IBM Spectrum Symphony Developer Edition. IBM Spectrum Symphony Developer Edition uses the consumer ID to associate applications and service packages. Only one application can be enabled within a consumer ID and service packages must be deployed to the same consumer ID.

Where used: Consumer

Required or optional: Required

Valid values: Alphanumeric string that does not include any special characters. The consumer ID begins with a forward slash (/) and each level is separated from the next with a forward slash.

Example:

consumerId="/NewYorkCluster/CapitalMarkets/Derivatives"

dataRequirement

Description: Stipulates whether SOAM workloads rely on data sets for grid synchronization.

If workloads rely on data sets, set the dataRequirement attribute to Dataset. Then, additionally, specify those data set names in the datasetDependencies attribute under the SessionTypes section of the application profile.

Where used: Consumer

Required or optional: Optional

Valid values: Dataset or None

Default value: None

Related attributes:

datasetDependencies in SessionTypes: Defines the names of the data sets that the workloads rely on for grid synchronization. Separate each data set name with a comma.

delaySlotRelease

Description: Specifies the amount of time, in seconds, that unassigned services will remain idle with the application before releasing their corresponding slot to EGO (and terminating the service). An unassigned service is defined as a service not assigned to any session. If new workload arrives before the configured time elapses, these unassigned services can be used to handle the new workload. Once an unassigned service becomes assigned to a session again, its idle timer is reset.

If delaySlotRelease is configured at the service-level, it overrides the application level setting for that service.

Note that delaySlotRelease is not supported for MapReduce workload.

Where used: Consumer

Required or optional: Optional

Valid values: 1 to 2147483647

Default value: 0

Related attributes:

delaySlotRelease: Specifies the delaySlotRelease time at the service level

dockerVersions

Description: Specifies a comma-separated list of allowable Docker versions; if not specified or empty, any Docker version is permitted.

Where used: Consumer

Required or optional: Optional

Default value: empty

Related attributes:

enableDockerForServiceInstance: Enables Docker containers for service instances.

enableDockerForServiceInstance

Description: Enables Docker containers for service instances of an application.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

true: Specifies that service instances will run in Docker containers.
false: Specifies that service instances will not run in Docker containers.

Default value: false

Related attributes:

dockerVersions: Specifies allowable Docker versions.

enableIncrementalSlotRequest

Description: Enables or disables the incremental slot allocation feature. When this feature is enabled, the SSM requests EGO only for the number of slots it can use at a time, rather than the maximum number of slots based on the total number of tasks and the service to slot ratio. If the SSM requires more slots to complete its workload, the SSM requests EGO progressively for more slots. This feature is useful for sessions with a large number of short-running tasks.

Note that enableIncrementalSlotRequest is not supported for MapReduce workload.

This feature is not recommended when the global standby service feature is enabled.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

Related attributes:

slotRequestBase: Specifies the initial number of requested slots for an application.
slotRequestIncrement: Specifies the increase (in percentage) from the initial number of requested slots to the total number of requested slots.

enableGlobalStandbyServices (deprecated)

As of IBM Spectrum Symphony 7.3.1, this attribute has been deprecated.

For customers before IBM Spectrum Symphony 7.3.1, standbyServiceScope set to Consumer behaves the same as setting enableStandbyService in the Consumer section to enable the standby service feature in previous releases; standbyServiceScope set to Cluster behaves the same as setting enableGlobalStandbyServices to enable the global standby services feature in previous releases.

Description: Enables the global standby service feature, which allows one or more applications to keep service instances running in standby mode on released slots and to reattach to these service instances when required.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

Related attributes:

standbyServiceScope: Enables the standby service feature, which allows one or more applications to keep service instances running in standby mode on released slots and to reattach to these service instances when required. Standby services are applicable to both slot-based scheduling and multidimensional scheduling. Set the standbyServiceScope to Cluster to enable global standby services. Set the standbyServiceScope to Consumer to enable standby services.

enableMultiTaskService

Description: Enables the application to run multiple tasks concurrently on a service instance. Running multiple tasks concurrently in one service instance permits you to easily share data among tasks.

Where used: Consumer

Required or optional: Optional

Valid values: false, session, or application:

false: The Multiple Task Service feature is disabled.
session: Specifies that one service instance will share data among tasks for the same session.
application: Specifies that one service instance will share data among tasks for the same application.

Default value: false

enableResourceReservation

Description: Enables the application to reserve resources for the minimum services (R_MinimumServices) scheduling policy for data affinity.

This feature is an extension to the minimum service scheduling policy. To use this feature, you must first enable that scheduling policy (configured by specifying policy=R_MinimumServices in your application profile).

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

Related attributes:

resourceReservationTimeout in SessionTypes > Type: Specifies the maximum number of seconds that resrouces can be reserved by a session before the session times out. After this timeout period, the session aborts.
abortSessionWithResourceReservation in SessionTypes > Type: Specifies that when a task is re-queued for any reason, the session is aborted.

enableSelectiveReclaim

Description: Specifies whether selective reclaim is enabled. If set to true, this means selective reclaim is enabled. Selective reclaim allows you more control over which task is interrupted during resource reclaim, because the system uses preemption rank and preemption criteria (both configurable) to determine which host supplies the resource.

Without selective reclaim, the host is selected at random but the SSM uses preemption criteria and preemption rank to choose which slots are reclaimed on that host.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

enableServiceLevelBlockHost

Description: Specifies whether host blocking at the service level of an application is enabled. If set to true, the host can be blocked for a specific service of the SOAM application. Other services in the same application are free to use the host as normal.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

enableSlotRatioResourceLimitScaling

Description: Enables the scaling of memory and CPU limits based on ServiceToSlotRatio.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

true: Specifies that resource limit scaling is enabled.
false: Specifies that resource limit scaling is disabled.

Default value: false

enableStandbyService (deprecated)

As of IBM Spectrum Symphony 7.3.1, this attribute has been deprecated.

Description: Specifies whether resources that have been allocated to the application will have standby services running when slots are released by the SSM.

Global standby services (enableGlobalStandbyServices) and standby services (enableStandbyService) cannot be enabled together for an application. Only one feature can be configured at a time for an application. Note that standby services are not supported for MapReduce workload or by IBM Spectrum Symphony Developer Edition. In this case, standby service configuration is ignored by IBM Spectrum Symphony and a warning message is logged.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

Related attributes:

standbyServiceScope: Enables the standby service feature, which allows one or more applications to keep service instances running in standby mode on released slots and to reattach to these service instances when required. Standby services are applicable to both slot-based scheduling and multidimensional scheduling. Set the standbyServiceScope to Cluster to enable global standby services. Set the standbyServiceScope to Consumer to enable standby services.

enableStandbyCurrentService

Description: Specifies whether the SSM will put non-default services in standby mode. If this parameter is configure as true, the SSM will leave the most recently-used services running when it releases the slots back to EGO in standby mode.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

Related attributes:

standbyServiceScope: This parameter requires the standbyServiceScope parameter to set to Consumer.

enableTaskBulletin

Description: Enables or disables collection of task-level metrics on CPU time and memory usage, including physical memory and swap space, for SOAM and MapReduce workload.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

Related attributes:

collectTaskMetricsInterval: Specifies the interval (in seconds) that task-level metrics are collected on Linux hosts.

externalSocketsLibrary

Description: Enables usage of IBM's User Space Sockets (USS) library. Set the parameter to USS to enable library usage.

Where used: Consumer

Required or optional: Optional

Valid values: USS or None

Default value:None

flushDataAsap

Description: Used for recoverable sessions. Specifies whether or not the session manager caches data before writing to disk.

When set to true, data is not cached, it is immediately written to disk. When set to false, data is cached before it is written to disk. Note that the setting in flushDataAsap does not affect any operating system configuration. You need to disable write-behind caching for your operating system. On Windows, see http://support.microsoft.com/kb/247485 for more details. On Linux, contact your system administrator.

Important: Setting this parameter to true may substantially degrade performance because the session manager cannot continue with its next operation until the task data and session data (common data and common data updates, if applicable) are written to the disk.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

ioRetryDelay

Description: Specifies the amount of time to wait, in seconds, before retrying an I/O operation after a previous failure.

Where used: Consumer

Required or optional: Optional

Valid values: 1 to 2147483647

Default value: 1

keepBlockHostList

Description: Determines whether the block host list for an allocation of SSM or SIM is maintained if that allocation is replaced by a new allocation.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value:false

numOfSlotsForPreloadedServices

Description: Used with preStartApplication. Defines the number of slots required for service instances that are pre-started if preStartApplication is true.

Where used: Consumer

Required or optional: Optional

Valid values: 1 to 36000

Default value: 1

Related attributes:

preStartApplication: Services are preloaded only if preStartApplication is true
resourceGroupName: Defines the resources available to an application

policy

Description: Specifies the workload scheduling policy used. The scheduling policy determines how and when IBM Spectrum Symphony allocates resources to run services for the application. IBM Spectrum Symphony supports four workload scheduling policies:

Proportional scheduling policy, defined as policy="R_Proportion" in the Consumer sections of the application profile.
Priority scheduling, defined as policy="R_PriorityScheduling" in the application profile.
Minimum services scheduling policy, defined as policy="R_MinimumServices" in the application profile.
Tag-based share policy, defined as policy="R_TagHierarchy" in the application profile.

Note that if workload type is specified as MapReduce (workloadType="MapReduce") , the MapReduce scheduling policies are used.

Where used: Consumer

Required or optional: Optional

Default value: R_Proportion

Valid values:

Table 1. Scheduling policies
Scheduling Policy	Description
R_Proportion
Summary	A new session can get CPU slots or multidimensional scheduling metrics immediately without waiting for current sessions to finish. Sessions simultaneously share a portion of the total CPU slots assigned to the application. CPU slots or metrics are proportionally assigned to sessions based on priority. Note: This scheduling policy can have performance issues if it takes a long time to bind a service to a session since the service instances may toggle among open sessions.
Initial slot or metric allocation	IBM Spectrum Symphony assigns CPU slots or metrics first to higher priority sessions to ensure they finish faster. CPU slots or metrics are assigned proportionally to sessions according to their priority. CPU factors are taken into account during allocation. The most powerful idle CPU is allocated first to higher priority sessions. The CPU factor of the compute host is defined in the conf\vem_resource.conf file under the IBM Spectrum Symphony Developer Edition installation directory, and is detected by EGO in the IBM Spectrum Symphony cluster when IBM Spectrum Symphony starts on the host.
Slot or metric reallocation based on workload?	Yes. When a session does not need all slots or metrics allocated to it, the slots or metrics are released and reallocated to other sessions. Slots or metrics are only reallocated to sessions with pending tasks.
Can higher priority sessions take slots or metrics away?	Yes. To re-balance resources, as soon as a higher priority session needs more resources, IBM Spectrum Symphony can allocate slots or metrics that are being used by lesser priority sessions to the higher priority session after the tasks running on the slots or metrics are finished. Preemption refers to when a session is under-allocated, and it can preempt workload from other sessions to use those slots or metrics to finish the workload, rather than wait for the sessions to voluntarily release slots or metrics. If sessions use the same SlotDefinitionName value in the MDS section of the application profile, and preemption is enabled for the session (that is `preemtive=true` in the sessionTypes section), then sessions with the highest priority can preempt lesser priority sessions. If sessions use different SlotDefinitionNames, regardless of whether preemption is enabled for the session, sessions with the highest priority of one SlotDefinitionName can preempt lesser priority sessions of another SlotDefinitionName.
Service instance unbound from session?	Service instances are unbound from the session based on workload, when there are not enough pending tasks for all the service instances currently allocated.
R_PriorityScheduling
Summary	A new session with the highest priority can get as many CPU slots or multidimensional scheduling metrics as it needs. IBM Spectrum Symphony prioritizes sessions and then schedules workload based on that priority. For slot-based scheduling, IBM Spectrum Symphony all sessions are scheduled in a strict prioritized order. For multidimensional scheduling: The first nine sessions are scheduled in a strict prioritized order (that is, the highest prioritized session runs first, then the second prioritized session, and continues to the ninth prioritized session). The remaining sessions are grouped by slotDefinitionName. Sessions that use the same SlotDefinitionName value (as set in the MDS section of the application profile), are scheduled based on priority. Sessions that use different slotDefinitionNames are scheduled proportionally across sessions. CPU slots or metrics are only allocated to other sessions when the highest priority session does not have any pending tasks. If preemption is enabled, sessions with the highest priority can preempt lesser priority sessions. Following preemption, the resource is assigned to the highest priority session, and the task that was preempted is rerun.
Initial slot or metric allocation	IBM Spectrum Symphony assigns all available CPU slots or metrics first to the highest priority session to meet the demand and ensure the session finishes as soon as possible. If there are more slots or metrics available than the highest priority session needs, the session with the next highest priority gets the surplus slots or metrics. This pattern of overflow slot allocation continues, in turn, to other sessions in a descending priority order. For multidimensional scheduling, IBM Spectrum Symphony starts from the first nine sessions. If two sessions have equal priority, the session that was created first is considered to have the higher priority.
Slot or metric reallocation based on workload?	Yes. When a session no longer needs all slots or metrics allocated to it, the slots or metrics are reallocated to a session with the next highest priority. Slots or metrics are only reallocated to sessions with pending tasks.
Can higher priority sessions take slots or metrics away?	Yes. As soon as a higher priority session needs more resources, IBM Spectrum Symphony can allocate slots or metrics that are being used by lesser priority sessions to the higher priority session, after the currently running task is finished. For slot-based scheduling, if preemption is enabled (that is `preemtive=true` in the sessionTypes section), for the higher priority session, the current running task of the lesser priority session is preempted. For multidimensional scheduling. as soon as a higher priority session needs more resources, IBM Spectrum Symphony can allocate slots or metrics that are being used by lesser priority sessions to the higher priority session, after the currently running task is finished. Additionally, depending on the session, preemption happens in the following manner: For the first nine sessions, preemption will always happen across the sessions, regardless of the `preemtive` configuration set in the sessionTypes section. For the sessions after the first nine sessions (that is, starting from the tenth session), for sessions with different slot definitions, preemption will always happen. For sessions with the same slot definition, IBM Spectrum Symphony will honor the `preemtive` configuration set in the sessionTypes section.
Service instance unbound from session?	Service instances are unbound from the session when there are no pending tasks in the session.
R_MinimumServices
Summary	Use this scheduling policy when you are using common data so that service instances will be reused for tasks in the same session, eliminating the need to reload data for each task. With this scheduling policy, a minimum number of CPU slots or multidimensional scheduling metrics is allocated to a session regardless of workload or priority.
Initial slot or metric allocation	IBM Spectrum Symphony attempts to allocate slots or metrics to the session to satisfy the minimum configured number of service instances. If there are enough slots or metrics to meet the minimum number of instances for all open sessions: Minimum instances are allocated to sessions based on session submission time. Leftover slots or metrics are allocated to sessions proportionally based on session priority and submission time. If there are not enough slots or metrics to meet the minimum number for all sessions: Higher priority sessions get their minimum service requirements satisfied first. Sessions which do not have their minimum service requirement satisfied are still able to proceed. CPU factors are taken into account. CPU slots or metrics are evenly distributed among sessions based on speed so that all sessions get a proportion of the fastest CPUs.
Slot or metric reallocation based on workload?	Partial. A minimum number of CPU slots or metrics is always allocated to a session, regardless of workload. Additional CPU slots or metrics are distributed proportionally to other open sessions with pending tasks according to priority and submission time. When there is no workload in the session, CPU slots or metrics not serving the minimum required service instances are distributed proportionally to other open sessions with pending tasks according to priority and submission time. Reclaiming slots or metrics refers to when a session is not using the minimum required service instances, and a session reclaims those slots or metrics and uses them on other open sessions with pending tasks. For multidimensional scheduling, regardless of whether preemption is enabled for a session (that is regardless of whether `preemtive="true"` or `"false"` under the sessionTypes section in the application profile), sessions always reclaim slots or metrics under a minimum services scheduling policy. For example, within your application profile, if you set `policy="R_MinimumServices"` (under the Consumer section), and `minServices="2"` (under the SessionTypes section), even if the first session occupies all the slots or metrics, after the second session with the same SlotDefinitionName (defined under the MDS section) submits tasks, it will reclaim two slots or metrics, regardless of whether `preemptive="true"` or `"false"` in sessionTypes section. Sessions with no pending tasks do not receive any slots or metrics, except the required minimum service instances.
Can higher priority sessions take slots or metrics away?	Partial. A session always retains the required CPU slots or metrics to serve the minimum number of service instances, regardless of session priority. CPU slots or metrics serving the minimum number of service instances are not released. CPU slots or metrics not serving the minimum number of service instances are released and distributed to other sessions proportionally based on priority.
Service instance unbound from session?	The minimum number of service instances are always bound to the session until the session is suspended, killed, or closed. Service instances in addition to the minimum number are unbound from the session as needed, based on the priority and pending workload of all sessions.
R_TagHierarchy
Summary	Use this scheduling policy to distribute an application's resources among tags in a hierarchy based on the tag's relative position and predefined share ratios in the hierarchy. This policy is a share policy, and is useful when you want to limit the number of resources that each user can use. Within this share policy, a fair allocation means that if the workload of two users overlap, the slow-down percentage of the these users are equal. Within a resource demand, fairness is reached when the usage and capacity is proportionate to its share ratio compared with the siblings under the same consumer of the same share priority. Overall fairness is achieved when usage and capacity is satisfied at each level of the consumer hierarchy. Configure a scheduling tree using he Policy section of the application profile, including the setting for SchedulingOrder and SchedulingRules withn the Policy section. Within a scheduling tree, each level in the hierarchy represents a category of items, which is identifiable by a policy tag type. Scheduling hierarchies of up to five levels of policy tag types are supported. Order the policy tag types in the application profile to specify the order in which resources will flow down the tree. Each policy tag type is assigned a value at the time the tree is configured.
Initial slot or metric allocation	The session whose first-level tag (node) is the most under-allocated gets a resource. If the first-level tags of two sessions are equally under-allocated, the policy continues to search for the most under-allocated tags, as follows: The policy looks at the second level tags. The session whose second level tag is more under-allocated gets the resource first. If two sessions are equally under-allocated across (N-1) levels of tags, the session whose Nth-level tag is more under-allocated gets the resource first. If two sessions are equally under-allocated across all tags, the session itself that is more under-allocated gets the resource first.
Slot or metric reallocation based on workload?	Yes. As workload completes or new sessions are added or removed, the deserved number of resources that each session or tag is entitled to may change. By default, the SSM will let a running task run to completion before taking away the resource. At the next task dispatch opportunity, the resource is unassigned from the session if the session itself is over-allocated or any level in that session's tag path is over-allocated, provided that there is a less over-allocated session or less over-allocated tag that can make use of the resource. After the resource is unassigned, it is later reassigned to another session (possibly under another tag). Since the policy always tries to feed resources to the highest-level tag that is the most under-allocated, this may mean that the policy causes an already-under-allocated tag or session to become further under-allocated in exchange for raising the resource usage of an even more under-allocated tag or session.
Can higher priority sessions take slots or metrics away?	Yes. Preemption rank determines which sessions are to be preempted first and which sessions are eligible to be preempted at all. If preemption is enabled for a session (that is, `preemtive="true"` under the sessionTypes section in the application profile), under-allocated sessions will take turns preempting over-allocated sessions. A session that tries to preempt other sessions is known as a preemptor session. The preemptor session must always be under-allocated, although the tags in its tag path may be over-allocated, under-allocated, or balanced.
Service instance unbound from session	When there are not enough pending tasks for all the service instances currently allocated.

Related attributes:

SessionType

priority: Determines how are tasks are assigned resources.
preemptive: Determines whether higher priority sessions preempt lesser priority sessions.
minServices: Determines the minimum number of CPU slots or metrics required for sessions.

Consumer

taskHighWaterMark: Defines the maximum ratio of unprocessed tasks to CPU slots or metrics before more resources are requested. By default, taskHighWaterMark is set to 1.0, and cannot be modified.
taskLowWaterMark: Defines the minimum ratio of unprocessed tasks to CPU slots or metrics before unused resources are released. Valid values for taskLowWaterMark is 0.0 (to indicate SSM releases resources only when there are reclaim requests), or 1.0 (to indicate SSM releases resources if workload does not need it).
resourceBalanceInterval: Defines when IBM Spectrum Symphony next checks to determine whether resources need to be requested or released from EGO.
sessionSchedulingInterval: Determines when resources are balanced among sessions.

preemptionCriteria

Description: Specifies the policy used to decide which task to interrupt to reclaim a resource.

This setting affects the resource reclaim process when the system has identified multiple sessions or tasks that could provide a resource, even after considering preemption rank or session priority.

If the value is set to MostRecentTask, the system selects the task with the least run time, and reclaims the resource used by that task. If multiple tasks have the same run time, the system selects any one at random. If multiple tasks run on a slot, consider the cumulative run time of all tasks using the slot.

If the workload scheduling policy (session scheduling policy) is proportional or minimum services, and the value is set to PolicyDefault, the system selects the most over-allocated session, and reclaims a resource used by that session. If multiple sessions are equally over-allocated, the system selects any one at random. If no session is over-allocated, select the least under-allocated instead. Task selection is random.

If the workload scheduling policy (session scheduling policy) is priority, and the value is set to PolicyDefault, the system selects a task at random.

There is less system overhead if you choose PolicyDefault.

Where used: Consumer

Required or optional: Optional

Valid values: PolicyDefault or MostRecentTask

Default value: PolicyDefault

preemptionScope

Description: Determines whether sessions in the application can preempt sessions of lesser or equal rank, or only preempt sessions of low rank when preemptive is set to true.

Where used: Consumer

Required or optional: Optional

Valid values: LowerRankedSessions or LowerOrEqualRankedSessions

Default value: LowerOrEqualRankedSessions

Related attributes:

preemptive: Specifies whether a session can preempt other sessions when it has workload to run.

preStartApplication

Description: Specifies whether the session manager, service instance manager, and service instances are pre-started for enabled applications when IBM Spectrum Symphony is started, or when an application is enabled.

Pre-starting the session manager (or application manager for MapReduce workload) provides quicker response to the first client request for this application. Pre-starting the service instances manager and service instances provides quicker response to tasks.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

Related attributes:

numOfSlotsForPreloadedServices: Defines the number of service instance slots required for a pre-started application. The number of slots cannot be greater than the number of slots available in the resource group, defined by resourceGroupName

recursiveBackfillThreshold

Description: Defines a threshold for the number of backfill tasks that can run on a slot. Yielded tasks can return to Running state if no other tasks are running on the slot. If the number of backfill task in a slot reaches this threshold, IBM Spectrum Symphony will only choose a new backfill task from a child level that is less than any other backfill tasks in the parent-child hierarchy. The default threshold is 5.

Consumer

Required or optional: Optional

Valid values: 0 to 100

Default value:5

resourceAllocationType

Description: Specifies the resource allocation type to use for a resource plan. For example, for a slot-based resource plan, specify slot as the resource allocation type; for a multidimensional scheduling resource plan, specify MDS.

Where used: Consumer

Required or optional: Optional

Valid values: slot or MDS:

slot: Indicates that slot-based allocation should be used for scheduling resources in the resource plan.
MDS: Indicates that multidimensional allocation should be used for scheduling resources in the resource plan.

Default value: slot

Related attributes:

slotDefinition under MDS > SlotDefinitions: Specifies the slot definition name, and indicates whether this slot definition is the default one in the resource plan.

resourceBalanceInterval

Description: Minimum number of seconds between checks to determine whether the session manager should release unused resources or request additional resources for the application.

Where used: Consumer

Required or optional: Optional

Valid values: 1 to 2147483647

Default value: 5

Related attributes:

resReq: Defines the resources an application requires
resourceGroupName: Defines the resource groups available to an application
sessionSchedulingInterval: Determines when resources are balanced among sessions
taskHighWaterMark: Defines the maximum ratio of unprocessed tasks to CPU slots before more resources are requested. By default, taskHighWaterMark is set to 1.0, and cannot be modified.
taskLowWaterMark: Defines the minimum ratio of unprocessed tasks to CPU slots before unused resources are released. Valid values for taskLowWaterMark is 0.0 (to indicate SSM releases resources only when there are reclaim requests), or 1.0 (to indicate SSM releases resources if workload does not need it).

resourceGroupName

Description: Resource group from which resources are requested to run service instance managers and services to perform computations for clients. The resource group should contain compute hosts, not management hosts. The resource group name can contain multiple resource groups, for example rg1 rg2 rg3

Note: The resourceGroupName value is not applicable forIBM Spectrum Symphony Developer Edition, or to MapReduce jobs.

Where used: Consumer

Required or optional: Optional

Default value: "" which means all resource groups which are associated with the consumer will be used.

Related attributes:

numOfSlotsForPreloadedServices: Defines the number of slots required for service instances that are pre-started if preStartApplication is true. The number cannot be greater than the number of slots available in the resource groups, defined by resourceGroupName
resReq: Defines the resources an application requires
resourceGroupFilter: Defines a list of resource groups from which the session can use resources. This list either matches or is subset of the resource groups specified in resourceGroupName.

resourceLimitationLevel

Description: Specifies resource usage limitation specifications in the application profile. The parameter is ignored if multidimensional scheduling is disabled.

Where used: Consumer

Required or optional: Optional

Valid values: None, Service, or Application:

None: Resource usage limitation is disabled.
Service: Enables resource usage limitation for a single service instance.
Application: Enables resource usage limitation for all service instances in an application.

Default value: None.

resourcePreference

Description: Determines whether the SSM tries to reduce the fragmentation of a session's tasks across hosts. If resourcePreference is set to StackedSessions, the SSM will minimize the number of hosts that are selected to run workload for the session. This behavior is at the individual session level.

You can specify a value of resourcePreference=StackedSessions for SIM to dispatch tasks of the same session to the same host, as much as possible. Alternatively, the default value of resourcePreference=Default indicates the tasks and hosts are random, rather than stacked.

Where used: Consumer

Required or optional: Optional

Valid values: Default or StackedSessions

Default value:Default

resReq

Description: String that describes criteria for selecting resources allocated to a consumer for compute hosts.

Note: The resReq value is not applicable to IBM Spectrum Symphony Developer Edition.

Where used: Consumer

Required or optional: Optional

Default value: select (!mg)

Related attributes:

resourceGroupName: Defines the resources available to an application

Synopsis: "select(select_string)""select(select_string)order(order_string)"

Synopsis description: A resource requirement string describes the criteria for defining a set of resources for compute hosts.

Note: If you specify a resource requirement, ensure you indicate "select(!mg)" in addition to your resource requirement string. This is so that the system only selects compute hosts to run work, not management hosts.

The entire resource requirement string cannot contain more than 512 characters.

Synopsis options:

select(select_string): Specifies the criteria for selecting the resources. The selection string filters out the resources that do not meet the criteria, resulting in a list of one or more eligible resources.
The parentheses must be typed as shown.
order(order_string): Specifies the sort order for selecting the best resource from the list of eligible resources. The most appropriate resource is listed first, the least is listed last.
The parentheses must be typed as shown.

select synopsis: select(expression)select(expression operator expression)select((expression operator expression) operator expression)

select description: The selection string (resource_name operator value) is a logical expression used to select one or more resources to match one or more criteria. Any resource that satisfies the criteria is selected.

resource_name

The following resources can be used as selection criteria.

Static resources

Static resources are built-in resources that represent host information that does not change over time, such as the maximum RAM available to user processes or the number of processors in a machine. Most static resources are determined at start-up time, or when hardware configuration changes are detected.

Static resources can be used to select appropriate hosts based on binary architecture, relative CPU speed, and system configuration.

Note: The resources ncpus, ncores, nprocs, nthreads, maxmem, maxswp, and maxtmp are not static on Linux hosts that support dynamic hardware reconfiguration.

Index	Measures	Units	Determined by
type	host type	string	configuration
model	host model	string	configuration
hname	host name	string	configuration
cpuf	CPU factor	relative	configuration
server	host can run remote jobs	Boolean	configuration
rexpri	execution priority	nice(2) argument	configuration
ncpus	number of processors	processors	LIM
ndiks	number of local disks	disks	LIM
maxmem	maximum RAM	MB	LIM
maxswp	maximum swap space	MB	LIM
maxtmp	maximum space in /tmp	MB	LIM

CPU factor (cpuf)

The CPU factor is the speed of the host’s CPU relative to other hosts in the cluster. If one processor is twice the speed of another, its CPU factor should be twice as large. CPU factors are defined by the cluster administrator. For multiprocessor hosts, the CPU factor is the speed of a single processor.

Server

The server static resource is Boolean. It has the following values:

1 if the host is configured to run jobs from other hosts.
0 if the host is a client for submitting jobs to other hosts.

operator

The following operators can be used in selection strings. If you are using the selection string in an XML format, you must use the applicable escape characters in the XML Equivalence column. The operators are listed in order of decreasing precedence:

Operator	XML equivalent	Syntax	Meaning
!	Not applicable	!a	Logical NOT: 1 if a==0, 0 otherwise
*	Not applicable	a*b	Multiply a and b
/	Not applicable	a / b	Divide a by b
+	Not applicable	a+b	Add a and b
-	Not applicable	a-b	Subtract b from a
>	&gt;	a > b	1 if a is greater than b, 0 otherwise
<	<	a < b	1 if a is less than b, 0 otherwise
>=	&gt;=	a >= b	1 if a is greater than or equal to b, 0 otherwise
<=	<=	a <= b	1 if a is less than or equal to b, 0 otherwise
==	Not applicable	a == b	1 if a is equal to b, 0 otherwise
!=	Not applicable	a != b	1 if a is not equal to b, 0 otherwise
&&	&&	a && b	Logical AND: 1 if both a and b are non-zero, 0 otherwise
\|\|	Not applicable	a \|\| b	Logical OR: 1 if either a or b is non-zero, 0 otherwise

value

Specifies the value to be used as criteria for selecting a resource. The value can be numerical, such as when referring to available memory or swap space, or it can be textual, such as when referring to a specific type of host.

Example: Simple selection expression using static resources. The following example selects resources with total memory greater than 500 MB:

select(maxmem &gt; 500)

Example: Compound selection expression using static resources. The following example selects resources with total memory greater than 500 MB and total swap space greater than or equal to 300 MB:

select(maxmem &gt; 500 &amp;&amp; maxswp &gt;=300)

order synopsis: order(order_string)

order description: The order string acts on the results of a select string, sorting the selected resources to identify the most favorable resources, and eliminate the least desirable resources. Resources are sorted into ascending order based on the result of an arithmetical expression. The result orders the resources from smallest to largest.

order_string

Specify an arithmetic expression that expresses the desired outcome.

You can create an expression using operators and numbers.

Use the following operators:

+: add
–: subtract-
*: multiply
/: divide

Example: Order resources based on available swap space. The following example orders the selected resources based on their available swap space, but sorts by descending order, ordering hosts with the greatest amount of available swap space first:

order(0-swp)

schedulingAffinity

Description: Determines whether resource-aware scheduling for sessions, data-aware scheduling, or rack-aware scheduling for tasks is enabled. When set to ResourceAware, the SSM collects host attributes and evaluates them against a resource preference expression for the session. When set to DataAware, the SSM collects data attributes of services instances and hosts and evaluates them against a user-defined preference expression for each task. Setting the attribute to DataAware automatically enables resource-aware scheduling for sessions. When set to RackAware, the SSM (or application manager for MapReduce workload) queries user-defined Hadoop Distributed File System (HDFS) topology to estimate the distance between the host on which a task is running and other hosts that have the input data for that task and dispatches waiting map tasks to idle hosts with the closest affinity in HDFS.

Where used: Consumer

Required or optional: Optional

Valid values: ResourceAware, DataAware, RackAware, or None

Note: For MapReduce workload, None is not one of the valid values.

Default value: None

scheduleTaskToStartedSI

Description: Specifies whether a task must be sent to a started service instance (SI) first, instead of a restarted SI. Use this attribute to avoid scheduling tasks to restarted SIs when started SIs are available; a task sent to a restarted SI is not run until the SI is started, causing the task to be delayed.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

Related attributes:

preStartApplication: When you configure tasks to be scheduled to started SIs first, use the preStartApplication attribute to ensure a sufficient number of idle SIs. Otherwise, some sessions might overuse SIs while others do not get their deserved share of SIs.

schedulingAffinityPlanningInterval

Description: For internal system use only.

sessionSchedulingInterval

Description: Minimum number of milliseconds between checks to determine whether compute resources need to be reallocated among sessions. IBM Spectrum Symphony triggers resource scheduling between sessions after the sessionSchedulingInterval as well as when the first task of a new session is created.

The system reallocates resources according to application session priority, scheduling policy, and pending tasks.

Where used: Consumer

Required or optional: Optional

Valid values: 1 to 2147483647

Default value: 500

Related attributes:

policy: Defines how resources are shared.
priority: Determines how are tasks are assigned resources.
taskHighWaterMark: Defines the maximum ratio of unprocessed tasks to CPU slots before more resources are requested. By default, taskHighWaterMark is set to 1.0, and cannot be modified.
taskLowWaterMark: Defines the minimum ratio of unprocessed tasks to CPU slots before unused resources are released. Valid values for taskLowWaterMark is 0.0 (to indicate SSM releases resources only when there are reclaim requests), or 1.0 (to indicate SSM releases resources if workload does not need it).

slotRequestBase

Description: Specifies the initial number of requested slots for an application. This parameter is required if you have enabled the incremental slot allocation feature (enableIncrementalSlotRequest).

Where used: Consumer

Required or optional: Optional

Valid values: 1 to 2147483647

Default value: None

Related attributes:

enableIncrementalSlotRequest: Enables or disables the incremental slot allocation feature.
slotRequestIncrement: Specifies the increase (in percentage) from the initial number of requested slots to the total number of requested slots.

slotRequestIncrement

Description: Specifies the increase (in percentage) from the initial number of requested slots to the total number of requested slots, which is defined by slotRequestBase.

Where used: Consumer

Required or optional: Optional

Valid values: Any value between 0.0 and 1.0 (supports up to three decimal places). When set to 0.0, the total number of requested slots (slotRequestBase) never increases.

Default value: 0.2

Related attributes:

enableIncrementalSlotRequest: Enables or disables the incremental slot allocation feature.
slotRequestBase: Specifies the initial number of requested slots for an application.

slotsThresholdForStandbyServices

Description: Maximum number of slots that standby services can run on for the application. If the number of slots exceeds this threshold, IBM Spectrum Symphony will release the excess slots at the next scheduling interval, or at the most within 100 scheduling intervals (which is the duration that the standby service is allowed to stay idle before it is terminated). If sessionSchedulingInterval is set to the default 500 milliseconds, excess slots will be released within 50 seconds.

Note that slotsThresholdForStandbyServices is not supported for MapReduce workload because standby services are not supported within MapReduce.

Where used: Consumer

Required or optional: Optional

Valid values: Any integer from -1 to 36000

Default value: -1, to indicate an unlimited number of slots

Related attributes:

standbyServiceScope: Specifies whether resources that have been allocated to the application will have standby services running when slots are released by the SSM.

ssmBalanceAllocationBetweenHosts

Description: Balances the host at the application level. The SSM balance allocation between hosts feature is based on the preStartApplication feature and minimum services scheduling policy. With the preStartApplication feature, the SSM requests slots and starts service instances (SIs) as soon as the SSM is enabled. All SIs must be ready before submitting a session for this feature. When the minimum services scheduling policy allocates the slots to the sessions, it first selects the host that has the most idle slots in its list before tasks are dispatched to it.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

Related attributes:

preStartApplication: Services are preloaded only if preStartApplication is true.
policy: Defines how resources are shared.

ssmPerfLogInterval

Description: This value is used to log SSM performance information. When enabled, the SSM logs performance information in regular intervals.

The following example of the SSM performance log shows that eight SIMs and one client connected to the SSM at 2014-11-12 00:53:59.342 Eastern Standard Time:

2014-11-12 00:53:59.342 Eastern Standard Time WARN [3960:3184] root -
The SIM connection number is <8>. The client thread number is <1>

Where used: Consumer

Required or optional: Optional

Valid values: 1 to 2147483647

Important: Setting ssmPerfLogInterval to a value that is less than or equal to 0 is illegal and prevents the application profile from being registered.

Default value: None. If ssmPerfLogInterval is undefined, this feature is off.

Example: The following example demonstrates the configuration of the ssmPerfLogInterval attribute.

<Consumer applicationName="symping7.3.1"
          consumerId="/SymTesting/Symping731"
          numOfSlotsForPreloadedServices="1"
          preStartApplication="false"
          resourceGroupName="ComputerHost"
          ssmPerfLogInterval="10"
          taskHighWaterMark="1.0"
          taskLowWaterMark="1.0"/>

Related attributes:

numOfSlotsForPreloadedServices: Defines the number of slots that are required for service instances that are pre-started if preStartApplication is true.
preStartApplication: Services are preloaded only if preStartApplication is true.
resourceGroupName: Defines the resources available to an application.
taskHighWaterMark: Defines the maximum ratio of unprocessed tasks to CPU slots before more resources are requested. By default, taskHighWaterMark is set to 1.0, and cannot be modified.
taskLowWaterMark: Defines the minimum ratio of unprocessed tasks to CPU slots before unused resources are released. Valid values for taskLowWaterMark is 0.0 (to indicate SSM releases resources only when there are reclaim requests), or 1.0 (to indicate SSM releases resources if workload does not need it).

ssmRecoveryTime

Description: For internal system use only.

standbyOnPreferredRG

Description: Stipulates how a standby service starts for a preferred resource group.

Set the standbyOnPreferredRG attribute to true to specify that a standby service is restricted to be started only on hosts in the preferred resource group.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

standbyServiceScope

Description: Enables the standby service feature, which allows one or more applications to keep service instances running in standby mode on released slots and to reattach to these service instances when required. Standby services are applicable to both slot-based scheduling and multidimensional scheduling.

Set the standbyServiceScope to Cluster to enable global standby services. Set the standbyServiceScope to Consumer to enable standby services.

Note that standby services are not supported for IBM Spectrum Symphony Developer Edition or MapReduce workload. In this case, standby service configuration is ignored by IBM Spectrum Symphony and a warning message is logged.

Where used: Consumer

Required or optional: Optional

Valid values: Cluster, Consumer, None

Default value: None

Related attributes:

preStartApplication: Services are preloaded only if preStartApplication is true
resourceGroupName: Defines the resources available to an application
standbyServiceTag (in Service section): Specifies the cluster-wide tag that identifies the global standby service.

taskHighWaterMark

Description: Applies to the whole application. The ratio represents the total number of pending and running (unprocessed) tasks in open sessions to service instances. The value of taskHighWaterMark is fixed at 1.0 and cannot be modified; this means that for every unprocessed task, 1 CPU slot is requested (assuming the session has a service-to-slot ratio of 1:1). SSM always asks for resource where there is pending workload.

In IBM Spectrum Symphony Developer Edition, the number of CPU slots on a host is configured in the conf/vem_resource.conf file under the IBM Spectrum Symphony Developer Edition installation directory.

In IBM Spectrum Symphony, the number of CPU slots on a host is configured in the ResourceGroups.xml file through the cluster management console.

Where used: Consumer

Default value: 1.0 (not adjustable)

Related attributes:

resourceBalanceInterval: Defines when IBM Spectrum Symphony next checks to determine whether resources need to be requested or released from EGO
taskLowWaterMark: Defines the minimum ratio of unprocessed tasks to CPU slots before unused resources are released. Valid values for taskLowWaterMark is 0.0 (to indicate SSM releases resources only when there are reclaim requests), or 1.0 (to indicate SSM releases resources if workload does not need it).
serviceToSlotRatio: Defines the number of slots required to run a service instance

taskLowWaterMark

Description: Applies to the whole application. Ratio is used to determine whether idle CPU slots are released and made available for other applications to use. Once the ratio of unprocessed tasks to service instances is less than the taskLowWaterMark, resources are released and made available for other applications to use.

If taskLowWaterMark is set to 1.0, SSM releases resources if workload does not need it. Any resources the application releases must be idle. Take, for example, you have thousands of unprocessed tasks and 100 CPU slots are allocated to the Session Manager (or application manager for MapReduce workload). Assume the session has a service-to-slot ratio of 1:1. The following logic is true:

When the number of unprocessed tasks reaches 200, the ratio of unprocessed tasks to CPU slots is 200/100= 2. The Session Manager does not release any CPU slots.
When the number of unprocessed tasks reaches 100, the ratio of unprocessed tasks to CPU slots is 100/100=1. The Session Manager does not release any CPU slots.
When the number of unprocessed tasks is less than 100, the Session Manager releases slots until the ratio becomes greater than 1.

If taskLowWaterMark is set to 0.0, the application never returns resources voluntarily. SSM releases resources only when there are reclaim requests.

Where used: Consumer

Required or optional: Optional

Valid values: 0.0 or 1.0

Default value: 1.0

Related attributes:

resourceBalanceInterval: Defines when IBM Spectrum Symphony next checks to determine whether resources need to be requested or released form EGO
taskHighWaterMark: Defines the maximum ratio of unprocessed tasks to CPU slots before more resources are requested. By default, taskHighWaterMark is set to 1.0, and cannot be modified.
serviceToSlotRatio: Defines the number of slots required to run a service instance

transientDisconnectionTimeout

Description: Number of seconds the session manager waits for the client to reconnect when the connection between the client and session manager is broken.

When abortSessionIfClientDisconnect is true, sessions which are open on the client connection are aborted if a disconnected client does not reconnect to the session manager.

The transientDisconnectionTimeout counter is reset when a client connects or re-connects to the session manager. Reconnection is done by the IBM Spectrum Symphony client API, which is transparent to the client application.

Where used: Consumer

Required or optional: Optional

Valid values: 1 to 2147483647

Default value: 30

Related attributes:

abortSessionIfClientDisconnect: Specifies whether the session is aborted if the session manager detects that the connection between the client and the session manager is broken.

useMaxServicesForPriorityScheduling

Description: Specifies to use the maxServices configuration set in the SessionTypes section of the application profile with the R_PriorityScheduling scheduling policy. If this value is not set to true, then the system ignores the maxServices configuration for the R_PriorityScheduling scheduling policy.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

useTotalReclaimGracePeriodForSlotBackfill

Description: When a back-filled slot is reclaimed, SSM will go through all tasks for the back-filled slot (that is, child tasks and all parent tasks that yield the same slot). SSM then obtains the total of the reclaimGracePeriod values (as defined in the SessionTypes section of the application profile) for all of these tasks, and then uses this total as the final reclaimGracePeriod. This way, each task on the backfilled slot has adequate time to handle a reclaim request.

Where used: Consumer

Required or optional: Optional

Valid values: true or false

Default value: false

workloadType

Description: Type of workload specified by the application. Applies only to MapReduce workload.

Where used: Consumer

Required or optional: Required

Valid values: MapReduce

Default value:MapReduce