Generic batch orchestration

Generic batch orchestration provides capabilities to run a scheduler flow and invoke any IBM® SPSS batch job by taking inputs from a configurable XML file instead of developing separate message flows for a specific use case.

Generic batch orchestration also provides the flexibility of changing the scheduled time of a flow and web service input parameters through an XML file, without changing message flows.

In generic batch orchestration, an orchestration XML file is used to store all configurations. Message flows read the XML file at run time from the properties folder on the Enterprise Service Bus (ESB) node.

The following capabilities are available in generic batch orchestration.

AutoTrigger

AutoTrigger is used to automatically trigger scheduler flows. The AutoTrigger flow creates a TimeoutRequest message according to the scheduler configurations in the XML file, and places the message on the queue that is specified in the scheduler configuration. AutoTrigger can accept any changes that are made to scheduler configurations.

The queue to which the AutoTrigger flow places the TimeoutRequest message can either be SPSSJobIntegration.msgflow or any other custom flow that is configured to run at a scheduled time. To run at the scheduled time, SPSSJobIntegration.msgflow or a custom flow must contain an MQInput node, TimeoutControl node, and a TimeoutNotification node that is configured to match the Identifier and queue name configurations in the XML file.

The following figure shows the parameters that are required for a scheduler configuration.

Diagram of the required parameters for a scheduler configuration. Review the following information. — Figure 1. Required parameters for a scheduler configuration

The first trigger of the flow occurs on the date of the flow deployment or flow restart at the time specified (<scheduled_time></scheduled_time>) in the batch orchestration XML file. The trigger repeats at regular intervals, in days (<duration_in_days></duration_in_days>), as specified in the batch orchestration XML file.

If a change is made to duration_in_days, the change takes effect from the second run that was previously configured. For example, duration_in_days is set to 3, and with this value, the next time the flow is supposed to run is 2014-12-09. If duration_in_days is changed to 2, the change takes effect only after 2014-12-09. If you want the change to immediately take effect, you must restart the flow.

The following figure shows an example of the Timer flow.

Diagram of a timer flow example. In the example, HSTriggerQ goes to Timeout Control and EventErrorHandler. FHSTriggerQ goes to Timeout Control1 and EventErrorHandler. IHSTriggerQ fores to Timeout Control2 and EventErrorHandler, IHSTriggerQ goes to Timeout Control2 and EventErrorHandler. HSTrigger goes to MQ Output and Error_handler. FHSTrigger goes to MQ Output and Error_handler. IHSTrigger goes to MQ Output and Error_handler. — Figure 2. Timer flow example

SPSSJobIntegration

The SPSSJobIntegration flow is used to invoke the IBM SPSS SubmitJobWithOptions web service to trigger an SPSS job with parameters. The SPSSJobIntegration flow picks up parameters, the web service end-point URL, the SPSS JobLocation URL, and notificationEnabled field values from the XML file.

Parameters that are used in the web service can either be static parameters, which are values that are predefined in the XML file, or dynamic parameters, which are values that are expected to come from a data preparation flow. The data preparation flow is specific to a use case, or is a combination of both static and dynamic parameters. There can also be cases where there are no parameters that are required.

The type field in the XML file specifies whether the parameter is static or dynamic. The parameter name is configured in the name field. For static parameters, the parameter value is specified in the value field. For dynamic parameters, the value is taken from the input message in the data preparation flow, which is specific to a use case. In this case, the value field in the XML file contains the input message field name, from which the dynamic value is mapped.

For dynamic field mapping, the request from the data preparation flow must have Request as the parent XML tag name, and child elements must contain the parameters. This request must also contain Identifier as one of the child elements, which is used by SPSSJobIntegration to identify which set of parameters to use for the particular use case.

The following code is a sample request from a data preparation flow. StartDate is one of the dynamic parameter values.

<Request>
<Idenfifier>WTIMER</Identifier>
<StartDate>2014-02-02</StartDate>
</Request>

The following figure shows the message flow that SPSSJobIntegration uses.

Diagram of the SPSSJobIntegration message flow. — Figure 3. SPSSJobIntegration message flow

The following figure shows the parameters that the web service configuration requires.

Diagram of the required parameters for the web service configuration. — Figure 4. Required parameters for the web service configuration

To trigger SPSSJobIntegration, a data preparation flow or Timer flow node that is specific to the use case must be created. The remaining parameters and fields that are required to invoke the SPSS batch job are taken from the orchestration XML file.

If there are only static parameters that are required for an SPSS job invocation, AutoTrigger and SPSSJobIntegration flows can be used together as a combination. In scenarios where dynamic parameters are involved, an extra data preparation flow that prepares the data, as required by the use case, is also needed.

If a new feature orchestration is introduced in the orchestration XML file, the PMQBatchIntegration flow must be restarted.

Tip: If you want to change configurable properties in the XML file, edit the changes in a local copy of the file, and replace the run time XML file with the modified file to avoid problems that are due to file locking.

The following table describes the features that use batch orchestration to run a scheduler flow, an SPSS job, an external flow, or a combination of flows and jobs, based on the scheduler and web service configuration in the batch orchestration XML file. Depending on your needs, configurations for flows and jobs can be updated at run time in the XML file.

Table 1. Features that use batch orchestration
Feature name	Purpose
Parametric	Triggers the Parametric adapter invocation at the scheduled time once a day, and modifies the default Parametric SubUseCase name.
Inspection	Triggers the Inspection adapter invocation at the scheduled time once a day.
Warranty	Triggers the Warranty SPSS job at the scheduled time once a day through the web service.
HS training	Triggers the HS Training SPSS job at the scheduled time once every 90 days through the web service.
FBA training	Triggers the Feature-based Analytics training SPSS job at the scheduled time once every 90 days through the web service.
IHS training	Triggers the Integrated Health Score training SPSS job at the scheduled time once every 90 days through the web service.
TopNFailure	Triggers the TopNFailure feature at the scheduled time once a day, and invokes the SPSS job for TopNFailure Analytics and Event generation though the web service.
Maintenance	Triggers the Maintenance feature at the scheduled time once a day, and invokes the SPSS job for Maintenance Analytics and Event generation though the web service.
Distribution transformer current aging	Triggers the Distribution Transformer Current Aging SPSS job at the scheduled time once a day though the web service.
Distribution transformer projected aging	Triggers the Distribution Transformer Projected Aging SPSS job at the scheduled time once every 180 days though the web service.
Pole FBA	Triggers the Pole FBA SPSS job at the scheduled time once every 30 days though the web service.