Enabling multiple server partitions support by using the IBM MQ messaging provider

You can set up the batch environment to have servers function as batch dispatchers, while other servers function as batch executors.

Before you begin

Important: Each dispatcher server and executor must be configured to use the same, database-backed job repository.

About this task

Batch dispatchers accept requests from external clients and make them available to the batch executors. The batch executors receive requests that match its defined capabilities and execute those requests. If the job is configured to run partitions, the batch executors make them available to the other executors to run. The batch executors communicate by using Java Messaging Service (JMS). This task helps you configure the batch dispatch server and the batch executor by using the IBM MQ messaging provider.

The following procedure describes how to separately configure one executor that runs only jobs and another executor that runs only partitions. You can also configure an executor that runs both jobs and partitions, but this procedure describes the more specialized, fine-grained configuration that is required to configure the executors separately.

Procedure

Configure the batch JMS dispatcher.

Enable JMS support by adding the wmqJmsClient-2.0 feature to the feature manager in your server.xml file.

If you are using IBM MQ on z/OS in BINDINGS mode, ensure that you also add the zosTransaction-1.0 feature.
Add the batchJmsDispatcher element to your server.xml file on the server that hosts the batch dispatcher; for example:
```
<batchJmsDispatcher connectionFactoryRef={reference to a configured JMS connection factory}
	queueRef={reference to a configured JMS queue} />
```
Note: If you do not specify the connectionFactoryRef and queueRef attributes, the default value for connectionFactoryRef is batchConnectionFactory and the default value for queueRef is batchJobSubmissionQueue. You can specify the batchJmsDispatcher element as <batchJmsDispatcher/>. You still must configure the batchConnectionFactory JMS connection factory and the JMS batchJobSubmissionQueue queue in the server.xml file.

Add the corresponding JMS connection factory and JMS queue to the server configuration. This addition is not specific to batch configuration.

The following example illustrates the batch JMS dispatcher configuration and its JMS configuration by using IBM MQ bindings mode:


<batchJmsDispatcher connectionFactoryRef="batchConnectionFactory"
		      queueRef="batchJobSubmissionQueue" />

<!-- wmq resource adapter -->
<variable name="wmqJmsClient.rar.location"
   	value="${server.config.dir}/wmq.wlp.rar"/>

<!-- nativeLibraryPath is required for BINDINGS mode -->
<!-- the startup retry and reconnect retry properties are 
     recommended to ensure robustness of the system.-->
<wmqJmsClient nativeLibraryPath="/mqm/jms/java/lib" 
		startupRetryCount=999
		startupRetryInterval="1000ms"
		reconnectionRetryCount=10
		reconnectionRetryInterval="5m">
</wmqJmsClient>

<authData password="pwd" user="user">
</authData>

<jmsConnectionFactory id="batchConnectionFactory">
	<properties.wmqJms transportType="BINDINGS"
			   queueManager="WMQX">
	</properties.wmqJms>
</jmsConnectionFactory>

<!-- baseQueueName is the queue defined on WMQ system -->
<jmsQueue id="batchJobSubmissionQueue">
    <properties.wmqJms baseQueueName="BATCHQ"
			  priority="QDEF"
			  baseQueueManagerName="WMQX">
    </properties.wmqJms>
</jmsQueue>

The following example illustrates the batch JMS dispatcher configuration and its JMS configuration by using IBM MQ in client mode:


<batchJmsDispatcher connectionFactoryRef="batchConnectionFactory"
		      queueRef="batchJobSubmissionQueue" />

<!-- wmq resource adapter -->
<variable name="wmqJmsClient.rar.location"
   	value="${server.config.dir}/wmq.wlp.rar"/>

<!-- the startup retry and reconnect retry properties are 
     recommended to ensure robustness of the system.-->
<wmqJmsClient startupRetryCount=999
		 startupRetryInterval="1000ms"
		 reconnectionRetryCount=10
		 reconnectionRetryInterval="5m">
</wmqJmsClient>

<authData password="pwd" user="user">
</authData>

<jmsConnectionFactory id="batchConnectionFactory">
	<properties.wmqJms 
			   hostName="webs24.pok.stglabs.ibm.com"
			   transportType="CLIENT"
			   channel="WAS.JMS.SVRCONN"
			   port="1414"
			   queueManager="WMQX"/>>
	</properties.wmqJms>
</jmsConnectionFactory>

<!-- baseQueueName is the queue defined on WMQ system -->
<jmsQueue id="batchJobSubmissionQueue">
    <properties.wmqJms baseQueueName="BATCHQ"
			  priority="QDEF"
			  baseQueueManagerName="WMQX">
    </properties.wmqJms>
</jmsQueue>

Note: Ensure that you are referencing a jmsConnectionFactory element, as shown in the example, and not a jmsQueueConnectionFactory element. Using a jmsQueueConnectionFactory element does not work. Your configuration must include both a jmsConnectionFactory element and a jmsQueue element, but not a jmsQueueConnectionFactory element.

Configure the batch JMS executor for executing batch jobs and the dispatcher for dispatching partitions.
1. Enable JMS support by adding the wmqJmsClient-2.0 feature to the feature manager in your server.xml file.
2. Add the batchJmsDispatcher element to your server.xml file on the server that hosts the batch executor.
```
<batchJmsDispatcher connectionFactoryRef={reference to a configured JMS connection factory}
        queueRef={reference to a configured JMS queue} />
```
  Note: If you do not specify the connectionFactoryRef and queueRef attributes, the default value for the connectionFactoryRef attribute is batchConnectionFactory and the default value for the queueRef attribute is batchJobSubmissionQueue. You can specify the batchJmsDispatcher element as <batchJmsDispatcher/>. You still must configure the batchConnectionFactory JMS connection factory and the batchJobSubmissionQueue value in the server.xml file.
  
  Note: If you do not add the batchJmsDispatcher element to your server.xml file, the server does not dispatch partitions to run on multiple servers and runs the partitions locally.
3. Add the batchJmsExecutor element and a connectionFactory to your server.xml file on the server that hosts the batch executor. For more information, see step 1b and 1c.
```
<batchJmsExecutor activationSpecRef={configured activation specification}
	queueRef={reference to the configured JMS queue} />
```
  Note: If you do not specify the activationSpecRef and queueRef attributes, the default value for activationSpecRef is batchActivationSpec and the default value for queueRef is batchJobSubmissionQueue. You can specify the batchJmsExecutor element as <batchJmsExecutor/>. You still must configure the JMS activation specification for batchActivationSpec and the batchJobSubmissionQueue JMS queue in the server.xml file.
4. Add the corresponding JMS activation specification and JMS queue to the server configuration. This addition is not specific to batch configuration.
5. Define batch executor server capabilities by including a JMS message selector in the activation specification.
  - Filter based on system-defined properties:
    The following batch dispatcher properties are available on the batch JMS message that the batch executor can use to filter for inbound messages.
    - com_ibm_ws_batch_applicationName: the name of the batch application for the job request
    - com_ibm_ws_batch_moduleName: the module name of the batch application for the job request
    - com_ibm_ws_batch_componentName: the component name of the batch application for the job request
    - com_ibm_ws_batch_work_type: the work type : "Job"
    Note: Specify a message selector with at least the com_ibm_ws_batch_applicationName property to ensure that the executor receives only jobs that it can process.
    The following example indicates the messageSelector attribute for the executor to accept a job for the application SimpleBatchJob and BonusPayout:.
    messageSelector="com_ibm_ws_batch_applicationName = 'SimpleBatchJob' OR com_ibm_ws_batch_applicationName = 'BonusPayout'">
  - Filter based on user-defined properties:
    The batch dispatcher sets all job parameters that conform to the proper JMS message property on the batch dispatcher request message. These properties can also be used by the message selector to add extra filtering to the message selector. The property name, or identifier, must conform to JMS message property constraints. For example, the property is an unlimited length sequence of letters and digits, the first of which must be a letter. A letter is any character for which the method Character.isJavaLetter returns true, and includes '_' and '$'. A letter or digit is any character for which the method Character.isJavaLetterOrDigit returns true. Check the JMS API documentation for more information on JMS message selector.
    The following example illustrates a possible message selector by using the com_ibm_ws_batch_applicationName property and a job parameter specialCapability:
    messageSelector="com_ibm_ws_batch_applicationName = 'SimpleBatchJob' AND specialCapability = 'superCapability'">
  The following example illustrates the batch JMS executor configuration and its JMS configuration by using IBM MQ bindings mode:
```
<batchJmsExecutor activationSpecRef="batchActivationSpec"
		  queueRef="batchJobSubmissionQueue"/>


<variable name="wmqJmsClient.rar.location"
          value="${server.config.dir}/wmq.wlp.rar"/>



<wmqJmsClient nativeLibraryPath="/mqm/jms/java/lib" 
		startupRetryCount=999
		startupRetryInterval="1000ms"
		reconnectionRetryCount=10
		reconnectionRetryInterval="5m">
</wmqJmsClient>

<authData password="pwd" user="user">
</authData>

<jmsActivationSpec id="batchActivationSpec" >
    <properties.wmqJms destinationRef="batchJobSubmissionQueue" 
          destinationType="javax.jms.Queue"
          messageSelector="(com_ibm_ws_batch_applicationName = 'SimpleBatchJob' OR com_ibm_ws_batch_applicationName = 'BonusPayout') AND com_ibm_ws_batch_work_type='Job'""
          transportType="BINDINGS"
          queueManager="WMQX">
    </properties.wmqJms>
</jmsActivationSpec>


<jmsQueue id="batchJobSubmissionQueue">
    <properties.wmqJms baseQueueName="BATCHQ"
          priority="QDEF"
          baseQueueManagerName="WMQX">
    </properties.wmqJms>
</jmsQueue>
```
  The following example illustrates the batch JMS executor configuration and its JMS configurations by using IBM MQ client mode:
```
<batchJmsExecutor activationSpecRef="batchActivationSpec"
		  queueRef="batchJobSubmissionQueue"/>


<variable name="wmqJmsClient.rar.location"
          value="${server.config.dir}/wmq.wlp.rar"/>


<wmqJmsClient startupRetryCount=999  
		 startupRetryInterval="1000ms"
		 reconnectionRetryCount=10
		 reconnectionRetryInterval="5m">
</wmqJmsClient>

<authData password="pwd" user="user">
</authData>

<jmsActivationSpec id="batchActivationSpec" >
    <properties.wmqJms destinationRef="batchJobSubmissionQueue" 
			   messageSelector="com_ibm_ws_batch_applicationName = SimpleBatchJob' OR com_ibm_ws_batch_applicationName = 'BonusPayout'"
		 transportType="CLIENT"
		 channel="WAS.JMS.SVRCONN"
		 destinationType="javax.jms.Queue"
		 queueManager="WMQX"
		 hostName="webs24.pok.stglabs.ibm.com"
		 port="1414">
    </properties.wmqJms>
</jmsActivationSpec>


<jmsQueue id="batchJobSubmissionQueue">
    <properties.wmqJms baseQueueName="BATCHQ"
   	baseQueueManagerName="WMQX">
    </properties.wmqJms>
</jmsQueue>
```

Configure the batch JMS executor for executing only partitions.

Enable JMS support by adding the wmqJmsClient-2.0 feature to the feature manager in your server.xml file.

Add the batchJmsExecutor element and a connectionFactory to your server.xml file on the server that hosts the batch executor. For more information, see step 1b and 1c.

<batchJmsExecutor activationSpecRef={configured activation specification}
	queueRef={reference to the configured JMS queue}
       replyConnectionFactoryRef={reference to the configured JMS connection factory} />

The following table lists the default values for the activationSpecRef, queueRef, and replyConnectionFactoryRef attributes. If you do not specify any one of these attributes, its default value is applied.

Table 1. Default attribute values for the Batch JMS executor
Attribute	Default value
activationSpecRef	`batchActivationSpec`
queueRef	`batchJobSubmissionQueue`
replyConnectionFactoryRef	`batchConnectionFactory`

If you do not specify any of these attributes, you can specify the batchJmsExecutor element as <batchJmsExecutor/>. However, if you use the default values for these attributes, you must configure the JMS activation specification for the batchActivationSpec value, the batchJobSubmissionQueue queue, and the JMS connection factory for the batchConnectionFactory value in the server.xml file.

Add the corresponding JMS activation specification and JMS queue to the server configuration. This addition is not specific to batch configuration.

Define batch executor server capabilities by including a JMS message selector in the activation specification.

Filter based on system-defined properties:
The following batch dispatcher properties are available on the batch JMS message that the batch executor can use to filter for inbound messages.
- com_ibm_ws_batch_applicationName: the name of the batch application for the job request
- com_ibm_ws_batch_moduleName: the module name of the batch application for the job request
- com_ibm_ws_batch_componentName: the component name of the batch application for the job request
- com_ibm_ws_batch_work_type: the work type : "Partition"
- com_ibm_ws_batch_partitionNum(Type=Integer): the partition number of the partition that is being dispatched
- com_ibm_ws_batch_stepName: the name of the step as defined in the JSL
Note: Specify a message selector with at least the com_ibm_ws_batch_applicationName property to ensure that the executor receives only jobs that it can process.
The following example indicates the messageSelector attribute for the executor to accept a job for the application SimpleBatchJob and BonusPayout:.
```
messageSelector="com_ibm_ws_batch_applicationName = 'SimpleBatchJob' OR com_ibm_ws_batch_applicationName = 'BonusPayout'">
```
Filter based on user-defined properties:
The batch dispatcher sets all job parameters that conform to the proper JMS message property on the batch dispatcher request message. These properties can also be used by the message selector to add extra filtering to the message selector. The property name, or identifier, must conform to JMS message property constraints. For example, the property is an unlimited length sequence of letters and digits, the first of which must be a letter. A letter is any character for which the method Character.isJavaLetter returns true, and includes '_' and '$'. A letter or digit is any character for which the method Character.isJavaLetterOrDigit returns true. Check the JMS API documentation for more information on JMS message selectory.
The following example illustrates a possible message selector by using the com_ibm_ws_batch_applicationName property and a job parameter specialCapability:
```
messageSelector="com_ibm_ws_batch_applicationName = 'SimpleBatchJob' AND specialCapability = 'superCapability'">
```

The following example illustrates the batch JMS executor configuration and its JMS configuration by using IBM MQ bindings mode:


<jmsConnectionFactory id="batchConnectionFactory"/>

<batchJmsExecutor activationSpecRef="batchActivationSpec"
		  queueRef="batchJobSubmissionQueue"/>
               replyConnectionFactoryRef="batchConnectionFactory"/>

<!-- wmq resource adapter -->
<variable name="wmqJmsClient.rar.location"
          value="${server.config.dir}/wmq.wlp.rar"/>

<!-- nativeLibraryPath is required for BINDINGS mode -->
<!-- the startup retry and reconnect retry properties are
     recommended to ensure robustness of the system.-->
<wmqJmsClient nativeLibraryPath="/mqm/jms/java/lib" 
		startupRetryCount=999
		startupRetryInterval="1000ms"
		reconnectionRetryCount=10
		reconnectionRetryInterval="5m">
</wmqJmsClient>

<authData password="pwd" user="user">
</authData>

<jmsActivationSpec id="batchActivationSpec" >
    <properties.wmqJms destinationRef="batchJobSubmissionQueue" 
          destinationType="javax.jms.Queue"
          messageSelector="(com_ibm_ws_batch_applicationName = 'SimpleBatchJob' OR com_ibm_ws_batch_applicationName = 'BonusPayout') AND com_ibm_batch_work_type='Job'""
          transportType="BINDINGS"
          queueManager="WMQX">
    </properties.wmqJms>
</jmsActivationSpec>

<!-- baseQueueName is the queue defined on WMQ system -->
<jmsQueue id="batchJobSubmissionQueue">
    <properties.wmqJms baseQueueName="BATCHQ"
          priority="QDEF"
          baseQueueManagerName="WMQX">
    </properties.wmqJms>
</jmsQueue>

The following example illustrates the batch JMS executor configuration and its JMS configurations by using IBM MQ client mode:


<jmsConnectionFactory id="batchConnectionFactory"/>

<batchJmsExecutor activationSpecRef="batchActivationSpec"
		  queueRef="batchJobSubmissionQueue"/>
               replyConnectionFactoryRef="batchConnectionFactory"/>

<!-- wmq resource adapter -->
<variable name="wmqJmsClient.rar.location"
          value="${server.config.dir}/wmq.wlp.rar"/>

<!-- the startup retry and reconnect retry properties are
     recommended to ensure robustness of the system.-->
<wmqJmsClient startupRetryCount=999  
		 startupRetryInterval="1000ms"
		 reconnectionRetryCount=10
		 reconnectionRetryInterval="5m">
</wmqJmsClient>

<authData password="pwd" user="user">
</authData>

<jmsActivationSpec id="batchActivationSpec" >
    <properties.wmqJms destinationRef="batchJobSubmissionQueue" 
			   messageSelector="com_ibm_ws_batch_applicationName = SimpleBatchJob' OR com_ibm_ws_batch_applicationName = 'BonusPayout'"
		 transportType="CLIENT"
		 channel="WAS.JMS.SVRCONN"
		 destinationType="javax.jms.Queue"
		 queueManager="WMQX"
		 hostName="webs24.pok.stglabs.ibm.com"
		 port="1414">
    </properties.wmqJms>
</jmsActivationSpec>

<!-- baseQueueName is the queue defined on WMQ system -->
<jmsQueue id="batchJobSubmissionQueue">
    <properties.wmqJms baseQueueName="BATCHQ"
   	baseQueueManagerName="WMQX">
    </properties.wmqJms>
</jmsQueue>

Install your batch application on the server. For more information, see Deploying applications in Liberty.
Note: If you configure multiple server partition support for your server, but want to disable multiple server partition execution for an individual job, you can set the com.ibm.websphere.batch.partition.multiJVM job property to false in your JSL Job XML file. The following example illustrates a JSL job to disable multiple server partitions:
```
<job id="SimpleBonusPayoutJob" xmlns="http://xmlns.jcp.org/xml/ns/javaee"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/jobXML_1_0.xsd" version="1.0">
    <properties>
       <property name="com.ibm.websphere.batch.partition.multiJVM" value="false"/> 
    </properties>
```
Note: In versions 20.0.0.7 and earlier, if you are running a partition on a remote executor, no job logs are created for that partition.
In versions 20.0.0.8 and later, job logs are created for partitions that run on a remote executor.