Flat File Connectors
You can use the Flat File connector to translate documents into and from flat file formats. To set up the translation, you create the definition and structure of the Flat File connector, which is called a flat file schema. The schema also contains the instructions for parsing or creating the flat file and defines how to identify individual records within a flat file and what data is contained in each of those records.
What is a flat file definition and structure?
The definition and structure of a flat file connector contains the instructions for parsing or creating a flat file. It details the structure of the document, including delimiters, records, and repeated record structures. It also acts as the model against which you can validate an inbound flat file. A flat file structure consists of hierarchical elements that represent each record, field, and subfield in a flat file.
What are the different approaches to create a flat file connector?
You can create a flat file connector using any one of the following approaches:
- Create manually: In this approach, you define the definition and structure of a flat file connector and then manually add the elements or properties.
- Use a sample file: In this approach, you use a sample file to define the definition and structure of a flat file connector. Here, you use the automated wizards to create the structure of the flat file.
What are the high-level steps to create a flat file connector?
- Define the connector details: In this stage, you define the connector details and the approach. You can create the flat file connector either manually or by using a sample file.
- Define the record parser and specify a record identifier: In this stage, you associate a record parser with the flat file connector that processes the flat files inbound to IBM® webMethods Integration. You also specify how you want the record to be identified after it is parsed.
- Define the flat file structure: If you are creating the flat file connector manually, in this stage, you specify the hierarchical structure of the flat file by creating and nesting the record definitions.
Creating a flat file connector manually
IBM webMethods Integration can process flat files in which:
-
The records in the flat file are defined using one of the following methods:
Property Description Delimiter Each record in the flat file is separated by a delimiter. Fixed length Each record is a fixed number of bytes (for example, mainframe punch or print records). Variable length Each record is preceded by two bytes that indicate the length of the record. Records in the flat file can have different lengths. -
If the flat file contains record identifiers, they must be in the same location in all records in the file. IBM webMethods Integration can then identify fields in these records based on:
- Fixed position - Each field is defined by 1) the number of bytes from the beginning of the record and 2) the field length. This can be used regardless of whether a field delimiter has been specified.
- Delimiters - Each field is separated by a delimiter, and you can specify the Nth delimited field in a record to represent the record identifier. This can be used only when a field delimiter (and, if necessary, subfield delimiter) has been specified.
To create a flat file connector manually
-
From the IBM webMethods Integration navigation bar, click Projects.
-
Select a project and then click Connectors > Flat File > Add Connector.
The New Flat File Connector page appears.
-
Select Create manually and click Next.
-
In the Define Connector page, type a name and description of the flat file connector.
-
Drag and dropor click Browse files and select a connector icon, if necessary.
-
Click Next.
-
In the Flat File Definition page, configure the Parser and the Record Identifier.
-
In the Parser section, to configure a delimited record parser, select Delimiter in the Parser area. Use the Delimiter record parser for the Flat File connector when each record is separated by a delimiter. You can type the input or use the custom dropdown input with predefined list of values to select values for Record, Field or Composite, Sub Field, Quoted Release Character & Release Character fields. For a record delimiter, you can specify a character (for example, !) or character representation (for example, \r\n for carriage return).
Specify the following fields:
Delimiter Record type Description Record Character Character that separates records in a flat file document. --OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the record delimiter for this document is located.For example, if you specify 3 as the character position, you have indicated that the record delimiter appears in the fourth character position from the beginning of the document. Delimiter Field or composite type Description Field or Composite Character Character that separates fields in a flat file document. --OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the field delimiter for this document is located.For example, if you specify 4 as the character position, you have indicated that the field delimiter appears in the fifth character position from the beginning of the document. Delimiter Subfield type Description Subfield Character Character that separates subfields in a flat file document. The default is a period “.”. --OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the subfield delimiter for this document is located.For example, if you specify 5 as the character position, you have indicated that the subfield delimiter appears in the sixth character position from the beginning of the document. Delimiter Quoted Release Character type Description Quoted Release Character Character Character used to enable a section of text within a field to be represented as its literal value. Any delimiter characters that appear within this section will not be treated as delimiters.For example, your field delimiter is (,) and your release character is “. When you want to use (,) within a field as text, you must prefix it with your quoted release character. When using the convertFlatFileToDocument operation to create the strings Doe,JohnandDoe, Jane, the record would appear as “Doe, John”,“Doe, Jane”. When using the convertDocumentToFlatFile operation to create “Doe, John”,“Doe, Jane”, the value of the record would beDoe, John and Doe, Jane. When using the convertDocumentToFlatFile operation, if you have specified both the Release Character and the Quoted Release Character, the Quoted Release Character will be used.--OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the quoted release character for this document is located.For example, if you specify 5 as the character position, you have indicated that the quoted release character appears in the sixth character position from the beginning of the document. Delimiter Release character type Description Release Character Character Character used to enable a delimiter to be used for its intended, original meaning. The character following the release character will not be treated as a delimiter.For example, your field delimiter is + and your release character is \. When using + within a field as text, you must prefix it with your release character. When using the convertFlatFileToDocument operation to create the strings a+b+c and d+e+f, the record would appear as a\+b\+c+d\+e\+f. When using the convertDocumentToFlatFile operation to create a\+b\+c+d\+e\+f, the value of the record would be a+b+c and d+e+f. --OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the field delimiter for this document is located.For example, if you specify 5 as the character position, you have indicated that the field delimiter appears in the sixth character position from the beginning of the document. Next, set the record identifier.
-
Use a Fixed Length record parser type when each record is of a fixed length (for example, mainframe punch or print records). This parser splits a file into records of the same pre-specified length. To configure a fixed length record parser, in the Parser Type area, select Fixed Length.
Specify the following fields:
Fixed Length Description Record Length In the Record Length field, enter the length, in characters, of each record in the flat file. Record length cannot be empty. Fixed Length Field or composite type Description Field or Composite Character Character that separates fields or composites in a flat file document. --OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the field delimiter for this document is located.For example, if you specify 4 as the character position, you have indicated that the field delimiter appears in the fifth character position from the beginning of the document. Fixed Length Subfield type Description Subfield Character Character that separates subfields in a flat file document. --OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the subfield delimiter for this document is located.For example, if you specify 5 as the character position, you have indicated that the subfield delimiter appears in the sixth character position from the beginning of the document. Fixed length Quoted release character type Description Quoted Release Character Character Character used to enable a section of text within a field to be represented as its literal value. Any delimiter characters that appear within this section will not be treated as delimiters.For example, your field delimiter is (,) and your release character is “. When you want to use (,) within a field as text, you must prefix it with your quoted release character. When using the convertFlatFileToDocument operation to create the strings Doe,JohnandDoe, Jane, the record would appear as “Doe, John”,“Doe, Jane”. When using the convertDocumentToFlatFile operation to create “Doe, John”,“Doe, Jane”, the value of the record would beDoe, John and Doe, Jane.When using the convertDocumentToFlatFile operation, if you have specified both the Release Character and the and the Quoted Release Character, the Quoted Release Character will be used.--OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the quoted release character for this document is located.For example, if you specify 5 as the character position, you have indicated that the quoted release character appears in the sixth character position from the beginning of the document. Fixed Length Release Character type Description Release Character Character Character used to enable a delimiter to be used for its intended, original meaning. The character following the release character will not be treated as a delimiter.For example, your field delimiter is + and your release character is \. When using + within a field as text, you must prefix it with your release character. When using the convertFlatFileToDocument operation to create the strings a+b+c and d+e+f, the record would appear as a\+b\+c+d\+e\+f. When using the convertDocumentToFlatFile operation to create a\+b\+c+d\+e\+f, the value of the record would be a+b+c and d+e+f. --OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the field delimiter for this document is located.For example, if you specify 5 as the character position, you have indicated that the field delimiter appears in the sixth character position from the beginning of the document. Next, set the record identifier.
The Variable Length record parser type expects each record to be preceded by two bytes that indicate the length of the record. Each record may be a different length.
To configure a variable length record parser, in the Parser area, select Variable length, and specify the following fields:Variable Length Field or Composite type Description Field or Composite Character Character that separates fields or composites in a flat file document. --OR-- Character Position Starting from the beginning of the document and counting from zero \(0\), the character position at which the field delimiter for this document is located. For example, if you specify 4 as the character position, you have indicated that the field delimiter appears in the fifth character position from the beginning of the document. Variable Length Subfield type Description Subfield Character Character that separates subfields in a flat file document. --OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the subfield delimiter for this document is located. For example, if you specify 5 as the character position, you have indicated that the subfield delimiter appears in the sixth character position from the beginning of the document. Variable Length Quoted Release Character type Description Quoted Release Character Character Character used to enable a section of text within a field to be represented as its literal value. Any delimiter characters that appear within this section will not be treated as delimiters. For example, your field delimiter is (,) and your release character is “. When you want to use (,) within a field as text, you must prefix it with your quoted release character. When using the convertFlatFileToDocument operation to create the strings Doe,JohnandDoe, Jane, the record would appear as “Doe, John”,“Doe, Jane”. When using the convertDocumentToFlatFile operation to create “Doe, John”,“Doe, Jane”, the value of the record would beDoe, John and Doe, Jane. When using the convertDocumentToFlatFile operation, if you have specified both the Release Character and the and the Quoted Release Character, the Quoted Release Character will be used.--OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the quoted release character for this document is located. For example, if you specify 5 as the character position, you have indicated that the quoted release character appears in the sixth character position from the beginning of the document. Variable Length Release Character type Description Release Character Character Character used to enable a delimiter to be used for its intended, original meaning. The character following the release character will not be treated as a delimiter.For example, your field delimiter is + and your release character is \. When using + within a field as text, you must prefix it with your release character. When using the convertFlatFileToDocument operation to create the strings a+b+c and d+e+f, the record would appear as a\+b\+c+d\+e\+f. When using the convertDocumentToFlatFile operation to create a\+b\+c+d\+e\+f, the value of the record would be a+b+c and d+e+f. --OR-- Character Position Starting from the beginning of the document and counting from zero (0), the character position at which the field delimiter for this document is located.For example, if you specify 5 as the character position, you have indicated that the field delimiter appears in the sixth character position from the beginning of the document. Next, set the record identifier.
-
Specify a Record Identifier.
When parsing a file, IBM webMethods Integration looks at a record and extracts an identifier out of the data and uses that identifier to connect the record definition with a particular record in the flat file. The name of the record definition must match the value obtained by the record identifier.
To set the record identifier for a definition, if the flat file contains a record identifier, select Yes in the Record Identifier area, and set the record identifier to one of the following values.
Note: The Record Identifier Position has default value selected as Start at field position.Record Identifier Value Description Start at field position Identifies the field in the record (counting from zero) that contains the identifier. Start at field Position identifiers use the value of the specified field as the record identifier. These identifiers count from zero (0).For example, if 2 is specified, the third field is used as the record identifier. Start at character position Identifies the character position in the record (counting from zero) where the record identifier is located. Start at character position record identifiers compare the value that occurs in the record, at the specified offset, to all the record names defined in the flat file structure. Note that the start at character position identifier cannot distinguish between all types of record names.For example, if you name records “Rec1” and “Rec,” some instances of “Rec1” may be identified as “Rec,” because “Rec1” begins with “Rec.” -
Click Next to define the Flat File Structure. Use the Flat File Structure page to add records to the flat file structure and define the hierarchical relationships between them.
The connector receiving the flat file uses the structure to read the flat file. This structural information identifies the parent-child relationships between different records in the flat file. By nesting record elements in the flat file structure (adding record elements to a record), you can represent the hierarchical structure of the data in the flat file.
Enable the Edit Mode in the Data fields panel on the Structure page to:
- add a new element using the "+" icon and select Record Definition from the Element type field to specify a name for the record definition. For more information see the Usage Notes under step 11 of creating-a-flat-file-connector-from-a-sample-file.
- quickly navigate within the schema using the right, left, up and down arrows,
- perform edit actions for example cut, copy, paste and delete fields to modify the fields of the element in the property panel within the file schema, and
- utilize the keyboard shortcuts for modifying the flat file schema fields.
Note: The Show Data feature on the Data fields panel is available while creating the flat file using the import file function initially and not when you create the flat file manually or edit an existing flat file. - To add a Composite Definition after you have added a record definition,
specify the following:
Extractor type Description Nth Field Field number in the record that contains the composite you want to extract. This pulls the subfield data from the composite. If you leave this property empty, the composite will not be extracted. To add a Field Definition after you have added a record definition, specify the following:Extractor type Description Fixed Position Counting from zero (0), indicates a fixed number of bytes to be extracted from a record. Position Nth field Counting from zero (0), indicates the field that you want to extract from the record. Extractor Type a value to indicate the position of the field that you want to extract from the record. This value cannot be null and must be an integer greater than or equal to zero (0). For example, if you type 1, the second field will be extracted. This option is available only if you specified a field delimiter when configuring the definition and structure of the Flat File connector. This extractor returns the field as a key–value pair. The key is the name of the field. The value is the String value of the field.
Note: Ensure that the Record identifier position and Extractor type field values are the same for Field Definitions, else you might encounter errors while running the connector. For example, if you have selected the Record identifier position as Start at character position, then the Extractor type must be set to Fixed position. If you have selected the Record identifier position as Start at field Position, then the Extractor type must be set to Nth position. -
Click Next, the Summary page displays the Overview tab which shows the details under the Description and the File Type and the Structure tab, which displays both the file schema and sample output.
-
Click Save, to create the flat file connector. You will be redirected to Flat File Connectors page.
You can Edit or Remove a flat file connector from the Flat File Connector page.
Creating a flat file connector from a sample file
When you create a flat file connector from a sample file, ensure that the format of the sample file is *.txt.
To create a flat file connector from a sample file
-
From the IBM webMethods Integration navigation bar, click Projects.
-
Select a project and then click Connectors > Flat File > Add Connector.
The New Flat File Connector page appears.
-
Select Import from file, and then Drag and drop or click Browse Files to upload the sample file from your computer.
-
Click Next.
-
In the Define Connector page, type a name and description of the flat file connector.
-
Drag and drop or click Browse files and select a connector icon, if necessary.
-
In the Preview pane, IBM webMethods Integration updates the sample file contents based on your selections.
Note: The Show Preview will be collapsed by default on the Description page. Click to view the file content. -
Click Next.
-
In the Flat File Definition page, configure the Parser and the Record Identifier. Select the values from the dropdown menu or provide the custom input for the Delimiter, Fixed Length & Variable Length parsers. If the custom input is not in the valid format an error is seen.
In the Parser panel, select one of the following to indicate how the data in the sample file is formatted:
Record Parser Description Delimiter Use this parser when each record is separated by a delimiter. Fixed Length Use a fixed length record parser when each record is of a fixed length (for example, mainframe punch or print records). This parser splits a file into records of the same pre-specified length. Variable Length This parser expects each record to be preceded by two bytes that indicate the length of the record. Each record may be of different length. Select Use first row as field labels to make the first row as header.
Note: Use first row as field labels checkbox is disabled until record length is selected. This option is not available in manual flat file creation or editing existing flat file.If you select Delimiter as the parser type, specify the parser properties in the record parser panel. IBM webMethods Integration updates the Preview contents based on your selections. Use the following table to specify the delimiters used in the sample file.
Property Description Record Character that separates records in a flat file document. If a new line character (\n) exists at the end of each record and the Record delimiter is not ending with \n, then IBM webMethods Integration cannot render the records properly under the Preview pane. Field or Composite Character that separates fields in a flat file document. Subfield Character that separates subfields in a flat file document. Quoted Release Character Character used to enable a section of text within a field to be represented as its literal value. Any delimiter characters that appear within this section will not be treated as delimiters. For example, your field delimiter is (,) and your quoted release character is “. When you want to use (,) within a field as text, you must prefix it with your quoted release character. When using the convertFlatFileToDocument operation to create the strings Doe,JohnandDoe, Jane, the record would appear as “Doe, John”,“Doe, Jane”. When using the convertDocumentToFlatFile operation to create “Doe, John”,“Doe, Jane”, the value of the record would beDoe, John and Doe, Jane. When using the convertDocumentToFlatFile operation, if you have specified both the Release Character and the Quoted Release Character, the Quoted Release Character will be used.Release Character Character used to enable a delimiter to be used for its intended, original meaning. The character following the release character will not be treated as a delimiter. For example, your field delimiter is + and your release character is \. When using + within a field as text, you must prefix it with your release character. When using the convertFlatFileToDocument operation to create the strings a+b+c and d+e+f, the record would appear as a\+b\+c+d\+e\+f. When using the convertDocumentToFlatFile operation to create a\+b\+c+d\+e\+f, the value of the record would be a+b+c and d+e+f. If you select Fixed Length as the parser type, specify the parser properties in the record parser panel. IBM webMethods Integration updates the Preview contents based on your selections. Specify the position of each field in Field separators. The Field separators must be comma separated integer values, for example, 5, 10, 15. You can set a value for Record length to adjust the record so that it appears correctly under the Preview pane. Record length supports only positive integer values between one and total number of characters in the file. and cannot be empty.
If you selected Variable Length as the parser type, specify the parser properties in the Record Parser panel. IBM webMethods Integration updates the Preview contents based on your selections. Use the following table to specify the delimiters used in the sample file.
Property Description Field Character that separates fields in a flat file document. Subfield Character that separates subfields in a flat file document. Quoted release character Character used to enable a section of text within a field to be represented as its literal value. Any delimiter characters that appear within this section will not be treated as delimiters. For example, your field delimiter is (,) and your quoted release character is “. When you want to use (,) within a field as text, you must prefix it with your quoted release character. When using the convertToValues service to create the strings Doe,JohnandDoe, Jane, the record would appear as “Doe, John”,“Doe, Jane”. When using the convertToString service to create “Doe, John”,“Doe, Jane”, the value of the record would beDoe, John and Doe, Jane. When using the convertToString service, if you have specified both the Release Character and the Quoted Release Character, the Quoted Release Character will be used.Release character Character used to enable a delimiter to be used for its intended, original meaning. The character following the release character will not be treated as a delimiter. For example, your field delimiter is + and your release character is \. When using + within a field as text, you must prefix it with your release character. When using the convertToValues service to create the strings a+b+c and d+e+f, the record would appear as a\+b\+c+d\+e\+f. When using the convertToString service to create a\+b\+c+d\+e\+f, the value of the record would be a+b+c and d+e+f. -
Specify a Record Identifier
When parsing a file, IBM webMethods Integration looks at a record and extracts an identifier out of the data and uses that identifier to connect the record definition with a particular record in the flat file. The name of the record definition must match the value obtained by the record identifier.
To set the record identifier for a definition, if the flat file contains a record identifier, select Yes in the Record identifier area, and then set the record identifier to one of the following values.
Note: The Record Identifier Position has default value selected as Start at field position.Record Identifier Value Description Start at field position Identifies the field in the record (counting from zero) that contains the identifier. Start at field position identifiers use the value of the specified field as the record identifier. These identifiers count from zero (0). For example, if 2 is specified, the third field is used as the record identifier. Start at character position Identifies the character position in the record (counting from zero) where the record identifier is located. Start at character position record identifiers compare the value that occurs in the record, at the specified offset, to all the record names defined in the flat file structure. Note that the Start at character position identifier cannot distinguish between all types of record names. For example, if you name records “Rec1” and “Rec,” some instances of “Rec1” may be identified as “Rec,” because “Rec1” begins with “Rec.” Select No if the flat file does not contain a record identifier.
-
Click Next to view the Flat File Structure.
You can add records to the flat file structure and define the hierarchical relationships between them. The application receiving the flat file uses the structure to read the flat file. This structural information identifies the parent-child relationships between different records in the flat file. By nesting record elements in the flat file structure (adding record elements to a record), you can represent the hierarchical structure of the data in the flat file.
Enable the Edit Mode and use the Data fields panel on the Structure page to:
- add a new element using the "+" icon and select Record Definition from the Element type field to specify a name for the record definition,
- quickly navigate within the schema using the right, left, up and down arrows,
- perform edit actions for example cut, copy, paste and delete fields and
- utilize keyboard shortcuts for modifying the flat file schema fields.
You can also modify the fields of the element in the property panel within the file schema once the Edit Mode is on.
Usage Notes: The "+" option within the Data Fields panel functions in two ways:
- If the selected element is a Field, it adds a sibling Field element to the end of the list.
- If the selected element is any parent Element, the + option adds a child element to that parent.
- The right arrow functionality is applicable only when a parent element is present below the selected element.
- For schema elements, only the add and paste options are enabled. The paste option adds the copied or cut element as a child element.
Select the Show Data to preview the sample data if you have created the flat file using the import feature.
The Show Data feature is available only while creating the flat file using the import file function initially and not when you create the flat file manually or edit an existing flat file.
Use the Expand or Collapse All option to view the schema properties for the flat file.
-
To add a Composite Definition after you have added a record definition, click the Element Type field and select Composite Definition. Specify a name for the record definition and click Save.
Extractor type Description Nth Field-Extractor Field number in the record that contains the composite you want to extract. This pulls the subfield data from the composite. If you leave this property empty, the composite will not be extracted. -
To add a Field Definition after you have added a record definition, click the "+" icon and select Field Definition in the Element type field.
Extractor Type Description Fixed Position Counting from zero (0), indicates a fixed number of bytes to be extracted from a record. Extractor Position Nth Field Counting from zero (0), indicates the field that you want to extract from the record. Extractor This option is available only if you specified a field delimiter when configuring the definition and structure of the flat file connector. This extractor returns the field as a key–value pair. The key is the name of the field. The value is the String value of the field.
-
Click Next, the Summary page appears where the Overview tab provides information such as the File name, Description, and Connector icon. Additionally, it shows File Type details, including Parser type and Parser fields values. The Structure tab on the same page shows both the file schema and sample data details for the output.
-
Click Save to create the flat file connector. You will be redirected to Flat File Connectors page.
You can Edit, or Remove a Flat File connector from the Flat File Connector page.
Flat File Predefined Operations
The following predefined Flat File operations are available:
convertFlatFileToDocument
Converts a flattened flat file data into a structured data, which will conform to the document type associated with the flat file connector.
Input Variables
| ffDataString | String. The flat file input with type of String. | |
| ffData | Object. The flat file input with type of InputStream or ByteArray. If both ffData and ffDataString are parsed, ffDataString takes precedence. | |
| ffIterator | Object. Optional. An object that encapsulates and keeps track of the input data during processing. It is used only when the iterate variable has been set to true. | |
| encoding | String. Optional. The encoding of the InputStream passed in to ffData. The default encoding is UTF–8. | |
| delimiters | Document. Optional. A document object that contains the segment terminator and the field and
subfield separators. If the delimiter is null, it will be located using the
information defined in the definition and structure of the Flat File connector. To specify a
delimiter, you can specify: |
|
| One character or character representation (for example, *, \n for line terminator, \t for tab) | ||
| The space character | ||
| Variable | Description | |
| record | String. Character used to separate records. If you want to specify the two–character carriage
return line feed (CRLF) characters, specify \r\n. |
|
| field | String. Character used to separate fields. | |
| subfield | String. Character used to separate subfields. | |
| release | String. Character used to ignore a record, field, or subfield delimiter in a field. If a release character occurs in a field or subfield before the delimiter, it will be prefixed with the release. | |
| quotedRelease | String. Character to use to ignore a record, field, or subfield delimiter in a field. If a quoted release character occurs in a field or subfield before the delimiter, it will be prefixed with quotedRelease before being written to the output string. The string is pre- and appended with the quoted release character. For example, if * is a delimiter, the field value is a*b, and the quoted release character is “, the string appears as “a*b”. | |
| iterate | String. Optional. Whether you want to process the input all at one time.Valid values are
true and false. |
|
| batchsize | String. Optional. Whether you want to process the specified number of records. The input value of this field is a positive integer. It is applicable only when the iterate field value is set to true. If you do not specify this field value, default value is set to 1. | |
| createIfNull | String. Optional. Whether to create the document object if all the fields are null. | |
| skipWhiteSpace | String. Optional. Whether white space at the beginning of records will be ignored. | |
| keepResults | String. Optional. Whether to return the parsed data. | |
| validate | String. Optional. Whether to return error messages that describe how ffData differs from the definition and structure of the Flat File connector. | |
| returnErrors | String. Optional. Whether to return the validation errors. | |
| maxErrors | String. Optional. The maximum number of errors that can be returned from one record. When the flat file parser encounters more than the maximum number of errors within a record, the parser will stop parsing and return the parsed data and errors processed up until that point. | |
| flags | String. Optional. Flags that you can set to govern convertFlatFileToDocument options. | |
| Variable | Description | |
| addRecordCount | String. Whether you want the operation to add an additional field (@record–count) to each parsed record in the resulting document object. | |
| detailedErrors | String. Whether you want detailed conditional validation error information. | |
| skipToFirstRecord | String. Whether you want the operation to wait until it finds the first valid record before reporting invalid records as errors. | |
| trimWhitespace | String. Whether you want the operation to delete any blank spaces at the beginning of fields, at the end of fields, or both. | |
| resultAsArray | String. Whether you want the operation to return the document object that represents the input flat file data as a document reference that can be mapped to the document types generated. |
Output Variables
| _dt | Document. The name of the output document is flatfile_application_name_dt, where flatfile_application_name is the name of the flat file application. For example, if the connector name is test, then the name of the output document will be test_dt. The structure of the output document will be similar to what you have defined in your Flat File connector. |
| isValid | String. Whether flat file contains validation errors. |
| errors | String. Optional. An array containing the validation errors, if any, that were found in ffData or ffDataString. |
| hasMore | String. Optional. Indicates when the iteration is terminated. Use this variable as condition
to break the batching loop.This is applicable only when the iterate input is set to
true.Valid values are:
|
convertDocumentToFlatFile
Converts a structured data conforming to the given document type structure associated with the connector, to the flattened flat file data.
Input Variables
| <flatfile_application_name>_dt | Document Structure conforming to the Document type created for the corresponding application of this operation. | |
| spacePad | String. Optional. How to position the records in the flat file. | |
| signalError | String. Whether to create errors in the output. | |
| noEmptyTrailing Fields | String. Whether trailing empty fields are to be removed from the output. Used only with records that have delimited fields. | |
| noEmptyTrailing SubFields | String. Whether trailing empty subfields are to be removed from the output. Used only with records that have delimited fields. If no value is specified for the noEmptyTrailingSubFields parameter,IBM webMethods Integration uses the value set for the noEmptyTrailingFields parameter. | |
| delimiters | Document. Optional. The separator characters used to construct the output string. To specify a delimiter, you can specify: - One character or character representation (for example, *, \n for line terminator, \t for tab) | |
| Value | Description | |
| record | String. Character to use to separate records. If you want to specify the two–character
carriage return line feed (CRLF) characters, specify \r\n. |
|
| field | String. Character to use to separate fields. | |
| subfield | String. Character to use to separate subfields. | |
| release | String. Character to use to ignore a record, field, or subfield delimiter in a field. If a release character occurs in a field or subfield before the delimiter, it will be prefixed with release before being written to the output string. | |
| quotedRelease | String. Character to use to ignore a record, field, or subfield delimiter in a field. If a quoted release character occurs in a field or subfield before the delimiter, it will be prefixed with quotedRelease before being written to the output string. The string is pre- and appended with the quoted release character. For example, if * is a delimiter, the field value is a*b, and the quoted release character is “, the string appears as “a*b”. | |
| FormatInfo | Document. Any values mapped to the FormatInfo variable will be passed unmodified to all format operations invoked by convertDocumentToFlatFile and convertFlatFileToDocument. | |
| Encoding | String. The type of encoding used to write data to the output file. The default encoding is UTF–8. | |
| sortInput | String. Optional. Whether you want the operation to sort the input records to match the definition and structure of the Flat File application. | |
| returnAsBytes | Returns the document as a string or as a byte array instead of a string. |
Output Variables
| string | String. Data that represents the flat file document. |
| bytes | Object. If the input variable returnAsBytes is true, returns the output as a
byte array encoded using the specified encoding. The string value is not returned. |
| errorArray | Object. String array containing messages pertaining to errors that occurred during conversion |
Example - Invoking Flow services for Flat File Connector
After creating the Flat File connector, let us see how to create a Flow service and invoke one of the following predefined operation:
- convertFlatFileToDocument - Converts the flat file to a document (inbound)
- convertDocumentToFlatFile - Converts a document to a flat file (outbound)
Before you begin
- Log in to your tenant and enable Flow services.
- Check if you have the Developer and Admin roles assigned from the Settings > Roles page.
- In IBM webMethods Integration, ensure you have the Flat File connector with defined connector details. For example, Delimited_flatfile_connector.
Basic Flow
-
Select the project where you want to create the new Flow service. You can also create a new project.
-
Click the Flow services tab and on the Flow services page, click the "+" icon.
-
Provide a name, for example, Flatfile_flowservice, and an optional description for the new Flow service.
-
Type Delimited_flatfile_connector in the search box, select it, and then select the operation, convertFlatFileToDocument.
-
Click the mapping icon to map the input and output fields.
- Click the pipeline input fields and enter values. The service output is automatically mapped to the pipeline output by a dotted line.
-
Save the Flow service and then run it.
The data is now converted into a structured data, which will conform to the document type associated with the flat file connector.
Usage Notes
When the convertDocumentToFlatFile operation executes, the field that is defined to start after the end of the fixed length record will not be included in the output data if the following conditions are met:
- The definition and structure of the Flat File connector uses a fixed length record delimiter.
- The definition and structure of the Flat File connector contains a fixed position field that begins beyond the defined length of the fixed length record.
- The input to the convertDocumentToFlatFile operation contains a value for the fixed position field that begins beyond the defined length of the fixed length record.