EndMapper v1 Chunking

The ISF for a batch has the following format:

<isf:ISFMessages>
  <Batch>             <!-- 1:M -->
    <BatchHeader/>
    <isf:ISFMessage/> <!-- 1:M -->
  </Batch>
</isf:ISFMessages>

Normally, the input mapper would map the entire batch into a single ISFMessages message. The EndMapper would then process each transaction into the database and raise the transaction mapped events. When chunking is used, the input mapper must:

• Construct partial ISFMessages documents, which are called chunks.
• A good chunk size would be the AGGREGATION_THRESHOLD for the E_MpInTxnMapped event; for example, 100.
• Each chunk must contain a batch or batches.
• The last batch in the chunk does not need to be complete.
• The final chunk of a batch is identified by the presence of the BatchHeader element. Earlier chunks must not contain a BatchHeader.
• In a multiple batch ISF message, the last batch in the chunk must contain an element called MoreBatches in the BatchHeader with the value 'Y' until the mapper completes the final chunk of the final batch. Examples of Chunks shows an example of a multiple batch ISF message.
• For each chunk, the mapper should identify the transactions that failed mapping by setting the following environment values:
  • Environment.PMP.Variables.BATCH_MAPPING_VARS.MAPPING_RESULTS.SUCCESS[idx] = FALSE, where idx is the one-based index of the ISF message in the chunk.
  • Environment.PMP.Variables.BATCH_MAPPING_VARS.MAPPING_RESULTS.ERROR_CODE[idx] = xxx, where idx is the one-based index of the ISF message in the chunk, and xxx is any meaningful error code.

Examples of Chunks: This is a chunk of a batch, but not the last chunk of the batch:

<isf:ISFMessages>
  <Batch>             <!-- 1:M -->
    <!-- No Batch Header -->
    <isf:ISFMessage/> <!-- 1:M -->
  </Batch>
</isf:ISFMessages>

This is the last chunk of a batch, but not the final batch:

<isf:ISFMessages>
  <Batch>             <!-- 1:M -->
    <BatchHeader>
        <CustomerAssignedIdentifier>...</CustomerAssignedIdentifier>
        <TotalTransactionCount>...</TotalTransactionCount>
        <TotalTransactionAmount>...</TotalTransactionAmount>
        <MoreBatches>Y</MoreBatches>
    </BatchHeader>
    <isf:ISFMessage/> <!-- 1:M -->
  </Batch>
</isf:ISFMessages>

This is the last chunk of the final batch:

<isf:ISFMessages>
  <Batch>             <!-- 1:M -->
    <BatchHeader>
        <CustomerAssignedIdentifier>...</CustomerAssignedIdentifier>
        <TotalTransactionCount>...</TotalTransactionCount>
        <TotalTransactionAmount>...</TotalTransactionAmount>
        <MoreBatches/>  <!- this element may also be omitted -->
    </BatchHeader>
    <isf:ISFMessage/> <!-- 1:M -->
  </Batch>
</isf:ISFMessages>

When working with batch inbound mappers, keep the following advice in mind:

• Use a reference variable to reference the InputRoot tree so that individual transactions can be deleted once they have been mapped.
  Note: This supersedes previous advice to copy the message to the environment.
• Try to avoid look-forwards in a message where the parser may be forced to parse a large portion of the message to evaluate the statement. If the parser does look a long way forward, it is likely to cause a memory-related abend. A typical mistake here would be to look for the existence of the next batch when starting to process the current one.
• Never use the CARDINALITY() function to find the number of transactions prior to executing an index bound loop. Instead, loop using WHILE LASTMOVE(rTxn) and MOVE rTxn NEXTSIBLING. CARDINALITY() is both inefficient and a potential memory abend waiting to happen.