Business Process development and implementation Hints & Tips for better performance

Body

While developing your Business Processes (BP), there are topics you need to consider before implementing
them into your IBM Sterling B2B Integrator (ISBI) system.

The queue assigned to the BP, the persistence levels and the coding of the BPML itself can impact the way the BP executes.
It is important to note that the correct usage of these options can improve the overall execution performance of the BP.

This blog aims to provide some hints and tips on BP development and implementation to help you improve the performance
of your BP executions.
 

BP queues

By default all standard system BPs run on queue 4, also when you create a new BP, the default queue assigned is queue 4.
As you implement more and more BPs and if you always leave this as the default queue, it could lead to BP queue build up
and delays executing your BPs when there are many BPs running at the same time.





 

Review your queues configuration and threads assigned to each queue, make sure that you assign your custom BPs to the queues
best suited to their execution.

Alternatively please review the documentation below about queue tuning:
Queue tuning documentation

See also below a very useful presentation created by one of our experts that explains in detail the queues configuration parameters:
ISBI queue tuning presentation

It is recommended to have your system sized and tuned properly by an expert, please contact IBM services via your IBM account
manager if you need help sizing and tuning your system.
 

BP Persistence levels

It is important to assign the correct persistence level when creating a BP in your system.
By default with full persistence, every step executed by the BP is written to the database, so lowering the persistence
will increase performance of the BP and utilize less space in the database since fewer steps are written to the database tables.



 

Note that some services may not run properly when setting the persistence too low, so you need to test your BPs thoroughly
to make sure the persistence that best suits your needs is used.

See below links to the documentation about how to adjust the global persistence level and the individual BP persistence level:
Changing persistence levels
BP persistence options

If a certain service does not run correctly with a lower persistence, you can always try to override the persistence for that particular
service during runtime and see if it helps.
See below link to a technote that explains how to override the BP persistence during runtime for a certain service or adapter using the parameter WF_RUNTIME_OVERRIDE_PERSISTENCE_LEVEL:
How to override the persistence level for a specific service in bpml

In the example below, you can see the Get document info service is set to override the BP persistence to full:

<operation name="GetDocumentInfoService">
<participant name="GetDocumentInfoService"/>
<output message="xout">
<assign to="WF_RUNTIME_OVERRIDE_PERSISTENCE_LEVEL">PERSISTENCE_FULL</assign>
<assign to="." from="*"/>
</output>
<input message="xin">
<assign to="." from="*"/>
</input>
</operation>

AE execute cycle queue parameter (AE_ExecuteCycle) for communication BPs.

If you are transferring files via FTP, sFTP or other protocol, it is advisable to make sure that your BP runs in one go
and it is not returned to the queue during execution of the comms services or adapters, for example, if a BP is returned to
the queue between the execution of a FTP client list service and a ftp client get service, this could cause the FTP connection to
timeout before the BP gets execution time once more.

AE_ExecuteCycle determines the amount of steps the BP will run when it starts before being placed back into the queue if other
BPs are waiting, so if your comms BP has per example 8 or 9 steps, you could assign 10 to the AE_ExecuteCycle of a certain queue
and have the BP run on that queue.
Again, this parameter is covered in detail in the presentation below:
ISBI queue tuning presentation


Do not forget to use "End Sessions" for communications BPs.

It is very important to make sure that every session established by any sort of protocol is properly closed.
When using a "Client suite of adapters" for a certain protocol, like FTP client suite of adapters, make sure that you always
finish the connection with an "Client end session";

it is also very important and this is often forgotten to make sure that you use an onfault covering the comms steps that also includes the "Client end session", to make sure that when an error occurs during the transfer or any other step of the comms, the BP goes into the onfault and closes the session properly.

See below example of a BP that performs an FTP transfer with a correctly assign FTP Client end session at the end of the BP and another contained in the onfault attached to the sequence that performs the FTP steps.

Avoid BP looping with a large amount of steps.

When developing a BP that contains a loop, make sure that you avoid looping for a large amount of steps.
Make sure that you include a limit on the looping counter so that the next BP will follow with the rest of
the processing.

Take into account that if you are per example processing 100 items via a loop and each loop contains 20 steps, your BP will have at least a total of 2000 steps, so BP execution can easily grow very large if you are processing a lot of items via a loop.

If your BP is started by a routing rule, you can adjust the amount of messages passed to each BP started by the routing rule, this behavior is dictated by the mailbox.properties parameter maxMessagesPerProcess.
By default this parameter is set to -1, meaning that it will pass all messages available at that time to one single BP execution.
So, if you have 1000 messages in the mailbox, they will all be passed onto that one BP execution for processing.
If you set the maxMessagesPerProcess parameter to 100 for example, then the routing rule will call up 10 instances of your BP and assign 100
messages to each of them.

The best way, to change the maxMessagesPerProcess parameter is to add it to your customer_override.properties and use the mailbox
prefix, for example: mailbox.maxMessagesPerProcess=100
Note that you need to restart ISBI for the change to take effect.


Keep ProcessData (Instance Data) tidy.

Try to always keep the Process Data as small as possible, very large ProcessData can impact BP performance and should be avoided.

See below some tips to take into consideration when dealing with ProcessData:

1. When using looping in your BPs, make sure that you remove unwanted information from ProcessData at the end of each loop; you can use the release service to remove unwanted entries from ProcessData.            To speed up processing while looping, you could try to always use the first entry on your process list for the looping and then use the release service to remove that entry from the list at the end of the each loop; this will make process data become smaller and smaller has you process your item list and therefore achieve better performance.

For example, the release service below would remove the contents of node config from ProcessData.

<operation name="Release Service">
<participant name="ReleaseService"/>
<output message="ReleaseServiceTypeInputMessage">
<assign to="TARGET">/ProcessData/test/config</assign>
<assign to="." from="*"></assign>
</output>
<input message="inmsg">
<assign to="." from="*"></assign>
</input>
</operation>

2. Use the Message from Service option with "Allow Message write" to filter the output from a Service to the Process Data using xPath statements.

When you use a service to retrieve certain information like Get document info service, you can control the results and only write the info
you need into ProcessData instead of the full result set.

See below example of BPML service configuration that writes the full result set to ProcessData and example of ProcessData returned:

<operation name="GetDocumentInfoService">
<participant name="GetDocumentInfoService"/>
<output message="xout">
<assign to="." from="*"/>
</output>
<input message="xin">
<assign to="." from="*"/>
</input>
</operation>

<ProcessData>
<PrimaryDocument SCIObjectID="rhel65_templ:node1:150cd007207:3054018"/>
<BodyName>input_document.txt</BodyName>
<BodyLength>193</BodyLength>
<DocumentBodyLength>193</DocumentBodyLength>
<DocumentLength>193</DocumentLength>
<DocumentName>input_document.txt</DocumentName>
<DocumentId>rhel65_templ:node1:150cd007207:3054016</DocumentId>
<DocumentSubject/>
<DocumentCreateTime>2015-11-06 12:34:03.202</DocumentCreateTime>
<DocumentModifiedTime>2015-11-06 12:34:03.202</DocumentModifiedTime>
<DocumentLifeSpan>0 Minutes</DocumentLifeSpan>
<DocumentInitialId>rhel65_templ:node1:150cd007207:3054016</DocumentInitialId>
<DocumentMaxInlineBodySize>102400</DocumentMaxInlineBodySize>
<DocumentPurgeAfter>2015-11-06 12:34:03.213</DocumentPurgeAfter>
<DocumentStorageType>DATABASE</DocumentStorageType>
<DocumentWorkflowId>1761983</DocumentWorkflowId>
<Encrypted>false</Encrypted>
<DocumentType>0</DocumentType>
<DocumentContentType>application/octet-stream</DocumentContentType>
</ProcessData>

And now an example of BPML service configuration that writes only the info you need to ProcessData, in this case only the document name and size, notice the difference on the ProcessData returned when comparing with the previous example.

<operation name="GetDocumentInfoService">
<participant name="GetDocumentInfoService"/>
<output message="xout">
<assign to="." from="*"/>
</output>
<input message="xin">
<assign to="." from="//BodyName"/>
<assign to="." from="//BodyLength"/>
</input>
</operation>
<ProcessData>
<PrimaryDocument SCIObjectID="rhel65_templ:node1:150cd007207:3056496"/>
<BodyName>input_document.txt</BodyName>
<BodyLength>193</BodyLength>
</ProcessData>

3. Avoid placing the result set returned by a service or adapter into the root of ProcessData, always use a specific node for that result set.

Some nodes written to ProcessData can have impact on other subsequent services or adapters executions when placed into the root of ProcessData.

<operation name="GetDocumentInfoService">
<participant name="GetDocumentInfoService"/>
<output message="xout">
<assign to="." from="*"/>
</output>
<input message="xin">
<assign to="GetDocInfoResults" from="*"/>
</input>
</operation>

As you can see below, the service configuration above no longer writes the results to the root of ProcessData but under the node GetDocInfoResults,
This will also make it easier to remove from ProcessData since removing the node GetDocInfoResults with the release service will also remove everything
under that node.



<ProcessData>
<PrimaryDocument SCIObjectID="rhel65_templ:node1:150cd007207:3054018"/>
<GetDocInfoResults>
<BodyName>input_document.txt</BodyName>
<BodyLength>193</BodyLength>
<DocumentBodyLength>193</DocumentBodyLength>
<DocumentLength>193</DocumentLength>
<DocumentName>input_document.txt</DocumentName>
<DocumentId>rhel65_templ:node1:150cd007207:3054016</DocumentId>
<DocumentSubject/>
<DocumentCreateTime>2015-11-06 12:34:03.202</DocumentCreateTime>
<DocumentModifiedTime>2015-11-06 12:34:03.202</DocumentModifiedTime>
<DocumentLifeSpan>0 Minutes</DocumentLifeSpan>
<DocumentInitialId>rhel65_templ:node1:150cd007207:3054016</DocumentInitialId>
<DocumentMaxInlineBodySize>102400</DocumentMaxInlineBodySize>
<DocumentPurgeAfter>2015-11-06 12:34:03.213</DocumentPurgeAfter>
<DocumentStorageType>DATABASE</DocumentStorageType>
<DocumentWorkflowId>1761983</DocumentWorkflowId>
<Encrypted>false</Encrypted>
<DocumentType>0</DocumentType>
<DocumentContentType>application/octet-stream</DocumentContentType>
<GetDocInfoResults/>
</ProcessData>

4. When using DocToDOM to retrieve info from the primary document into ProcessData, if not required, avoid placing the full content of the document into ProcessData but rather simply retrieve the information required.

See blog below on how to use DocToDOM to retrieve a specific node from a document.
Working with DocToDOM xPath function


 

Avoid relative paths in xPath.

When using xPath assigns in your BP, very often we use shortcuts by using relative paths but this can have an impact on performance specially when dealing with large ProcessData sets.
It is always better, performance wise, to specify the full path instead of using a relative path.

xPath full path example:
<assign to="test" from="/ProcessData/customer/type/text()"></assign>

xPath relative path example:
<assign to="test" from="//type/text()"></assign>
 

Control ProcessData on invoking sub business processes

Very often we use the Invoke service to call another BP execution.
When calling a sub BP via the invoke service, you can control the amount of ProcessData that will be passed
to the child BP using the message_to_child function so that only what you need is passed to the new BP and not the
full contents of ProcessData.

When the message_to_child tag is present in ProcessData, a invoked BP will get only ProcessData nodes under message_to_child.
See below link to the Invoke service documentation that explains the usage of message_to_child.
Invoke Sub-Process service documentation
 

For a full webcast on BPML development best practices, please see the link below:

BPML development best practices.