Splitting files

The inbound event processing mode supports an optional file splitting feature, where the event file is split into several business objects, also known as chunks. Each business object is posted to the export separately. This reduces memory loading during event processing. File splitting is performed based on either a delimiter or on a file size specified in the SplitCriteria property.

The adapter provides SplitBySize and SplitByDelimiter classes for file splitting. Optionally, you can provide a custom file splitter class and use it by providing the class name in the SplittingFunctionClassName property.

Splitting files by size

The splitting size is set in the SplittingFunctionClassName property.

Chunks refer to the resulting files after the file is split. When chunking is enabled, each chunk of the file is posted to the export separately. The number of business objects that are specified in the PollQuantity property is posted to the export. For example, if the value for PollQuantity is 3, then:

The number of business objects polled is 3.

The number of business objects received by the export is 3.

The adapter does not reassemble chunked data. It provides the information about the chunked data for an external application to merge the chunks. The chunking information is set in the chunkInfo property, which is contained in the business object. This information includes the chunk size in bytes, and the event ID. An example of an event ID is:

AbsolutePathOfTheEventFileNameInLocalEventDirectory_/_yyyy_MM_dd_HH_mm_ss_SSS.
currentBONumber

With WebSphere® Adapter for FTP, version 7.5, the event ID does not contain the total business object count, and hence it is not part of the chunk information. Optionally, you can add the total business object count in the chunk information by using the Include total business object count in the ChunkInfo property. For more information, see Include total business object count in the ChunkInfo (includeBOCountInChunkInfo).

Splitting files by delimiter

Delimiters are specified values, used for splitting the event files. The delimiter is specified in the SplitCriteria property.

The following rules apply when the delimiter is used:

The specified delimiter must not be the same as any of the data contained within the business object. If it is the same, file splitting can produce incorrect results.
The delimiter must contain the exact value of new line representation in the event file. The platform specific newline characters are shown in the following table.
Table 1. Platform specific newline characters
Platform Newline character

Macintosh \r

Microsoft Windows \r\n

UNIX \n
Use of more than one delimiter must be separated by a semicolon (;). If the semicolon is part of the delimiter, you must represent this as \;. For example, if the delimiter is ##\;## then it is processed as ##;##, which means that the semicolon is part of the delimiter.
To skip content that is part of the delimiter, specify a double semicolon (;;) in front of it so that the content between the delimiter is skipped. For example, if the event file contains a business object in the following format and the delimiter is ##;;$$, then:
Name=Smith
Company=IBM
##this is the content that will be skipped by the adapter$$

The adapter considers ##$$ as the delimiter and skips "this is the content that will be skipped by the adapter."
The delimiter accepts any value and there are no restrictions. The following are valid delimiter examples:
- ####;\n;\n
- ####;$$$$;\n;####
- %%%%;$$$$$;#####
- \n;\n;$$$$
- ####\;####;\n;$$$$$
- \n;\n;\n
- ####;;$$$$
- \r
- \r\n
- $$$$;\r\n
If the delimiter is located at the end of the file, the SplitCriteria property uses END_OF_FILE to determine the physical end of the file.
When each business object record in an event file is separated by a valid delimiter and if there is no delimiter or an invalid delimiter for the last business object record, the adapter can still process the business object records.
During inbound processing and splitting of the event file based on a delimiter, assume the business object records present in an event file are separated by a delimiter. And the delimiter is present at the beginning of each record instead of end of the record. Therefore, the adapter considers that the delimiter is always present at the beginning of each record and processes them accordingly.
If the delimiter is not found in the file, the entire content of the file is processed as one business object.

Table 1. Platform specific newline characters
Platform	Newline character
Macintosh	`\r`
Microsoft Windows	`\r\n`
UNIX	`\n`

Example 1:

John Doe,123,Washington Ave,222-123-4567
Jane Smith,234,Washington Ave,222-123-4568

The separator is the end of line character. In this example you would specify \r\n for Windows, \r for Macintosh, and \n for UNIX.

Example 2:

John Doe
123 Washington Ave
222-123-4567
####
Jane Smith
234 Washington Ave
222-123-4568

The separator is ####.

Example 3:

ISAJohnDoe1*IBM************USA************
ISAJohnDoe2*IBM************USA************
ISAJohnDoe3*IBM************USA************

The separator or delimiter in this example is ISA and it is at the beginning of each record.