Mapping Large Input Batches
For input mappers, FTM provides the following approaches for mapping large batches:
- Fragmentation
- This is more applicable to very large batches. Typically, fragmentation may be required when the number of
transactions approaches 100,000 though it can also be considered for smaller batches. The main advantage of
fragmentation is that it allows concurrent mapping of multiple fragments.
For more information, see Design for large files
- Chunking
- Typically, chunking is used when the batch size is expected to be more than a few thousand transactions. However, it is good practice to code with chunking regardless so that the mapper is scalable from the beginning. This mechanism allows the mapper to propagate a subset (chunk) of the transactions (data and ISF) to EndMapper for insertion to the database. The EndMapper v2 document interface includes explicit support for chunking while EndMapper v1 works with ISFMessages document structure as described in EndMapper v1 Chunking.
- Transaction Mode
- When using the default settings for EndMapper the database inserts are done through insertable views, the
insertable views are implemented using instead of triggers. DB2®
z/OS® has limitations on the number of these triggers that may be
executed in a unit of work that essentially limit the number of FTM transaction
inserts to approximately 5800 per unit of work. It is possible to work around this restriction by changing the
'Transaction Mode' property of the compute nodes that do the inserts such that the inserts are committed more
frequently.The following core subflows provide promoted properties that if used in the context of an inbound transmission wrapper should be promoted and connected such that they may be set as one in the context of a specific deployment situation:
Sub flow Property name EndMapper EndMapper_TransactionMode EndMapper_v1 EndMapper_TransactionMode EndMapper_v2 EndMapper_TransactionMode PhysicalTransmissionFlow LogPT_TransactionMode PhysicalTransmissionFlowLight LogPT_TransactionMode The default value for these properties is automatic. After the properties are promoted and connected at a top level wrapper flow, the value can be changed to commit. Making this change will ensure that any inserts from LogPT and EndMapper nodes are committed when the flow exits them. It is intended that this is used with maps that have been implemented to use chunking and the map must propagate chunks with less than the approximately 5800 limit. This will result in each set of chunk inserts being committed together. Use of small chunk sizes with this setting changed to commit will have a negative affect on performance.