Creating a flat file connector manually

Learn how to create a flat file connector manually.

About this task

webMethods Integration can process flat files in which:

The records in the flat file are defined by using one of the following methods:

Property	Description
Delimiter	Each record in the flat file is separated by a delimiter.
Fixed length	Each record is a fixed number of bytes (For instance, mainframe punch or print records).
Variable length	Each record is preceded by two bytes that indicate the length of the record. Records in the flat file can have different lengths.

If the flat file contains record identifiers, they must be in the same location in all records in the file. webMethods Integration can then identify fields in these records based on:

Fixed position: Each field is defined by the number of bytes from the beginning of the record and its field length. This method works even if a field delimiter is not specified.
Delimiters: Each field is separated by a delimiter, and you can specify the Nth delimited field in a record to represent the record identifier. This method can be used only when a field delimiter (and, if necessary, subfield delimiter) has been specified.

Procedure

Click Projects.
Select a project and then click Connectors > Flat File > Add Connector.
The New Flat File Connector page appears.
Select Create manually and click Next.
In the Define Connector page, type a name and description of the flat file connector.
Drag or click Browse files and select a connector icon, if necessary.
Click Next.
In the Flat File Definition page, configure the Parser and the Record Identifier.

In the Parser section, to configure a delimited record parser, select Delimiter in the Parser area. Use the Delimiter record parser for the Flat File connector when each record is separated by a delimiter. You can type the input or use the custom dropdown input with predefined list of values to select values for Record, Field, or Composite, Sub Field, Quoted Release Character & Release Character fields. For a record delimiter, you can specify a character (For instance, !) or character representation (For instance, \r\n for carriage return).

Specify the following fields:

Delimiter	Record type	Description
Record	Character	Character that separates records in a flat file document.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the record delimiter for this document is located. For instance, if you specify 3 as the character position, you have indicated that the record delimiter appears in the fourth character position from the beginning of the document.

Delimiter	Field or composite type	Description
Field or Composite	Character	A character that separates fields in a flat file document.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the field delimiter for this document is located. For instance, if you specify 4 as the character position, you have indicated that the field delimiter appears in the fifth character position from the beginning of the document.

Delimiter	Subfield type	Description
Subfield	Character	Character that separates subfields in a flat file document. The default is a period “.”.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the subfield delimiter for this document is located. For instance, if you specify 5 as the character position, you have indicated that the subfield delimiter appears in the sixth character position from the beginning of the document.

Delimiter	Quoted Release Character type	Description
Quoted Release Character	Character	Character used to enable a section of text within a field to be represented as its literal value. Any delimiter characters that appear within this section will not be treated as delimiters. For instance, your field delimiter is (,) and your release character is “. When you want to use (,) within a field as text, you must prefix it with your quoted release character. When using the convertFlatFileToDocument operation to create the strings `Doe`, `John`, and `Doe, Jane`, the record appears as “`Doe, John”,“Doe, Jane`”. When using the convertDocumentToFlatFile operation to create “`Doe, John`”,“`Doe, Jane`”, the value of the record becomes `Doe, John and Doe, Jane`. When using the convertDocumentToFlatFile operation, if you have specified both the Release Character and the Quoted Release Character, the Quoted Release Character is used.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the quoted release character for this document is located. For instance, if you specify 5 as the character position, you have indicated that the quoted release character appears in the sixth character position from the beginning of the document.

Delimiter	Release character type	Description
Release Character	Character	Character used to enable a delimiter to be used for its intended, original meaning. The character following the release character will not be treated as a delimiter.For instance, your field delimiter is + and your release character is \. When using + within a field as text, you must prefix it with your release character. When using the convertFlatFileToDocument operation to create the strings a+b+c and d+e+f, the record appears as a\+b\+c+d\+e\+f. When using the convertDocumentToFlatFile operation to create a\+b\+c+d\+e\+f, the value of the record appears as a+b+c and d+e+f.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the field delimiter for this document is located. For instance, if you specify 5 as the character position, you have indicated that the field delimiter appears in the sixth character position from the beginning of the document.

Next, set the record identifier.

Use a Fixed Length record parser type when each record is of a fixed length (For instance, mainframe punch or print records). This parser splits a file into records of the same pre-specified length. To configure a fixed-length record parser, in the Parser Type area, select Fixed Length.

Specify the following fields:

Fixed Length	Description
Record Length	In the Record Length field, enter the length, in characters, of each record in the flat file. Record length cannot be empty.

Fixed Length	Field or composite type	Description
Field or Composite	Character	Character that separates fields or composites in a flat file document.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the field delimiter for this document is located. For instance, if you specify 4 as the character position, you have indicated that the field delimiter appears in the fifth character position from the beginning of the document.

Fixed Length	Subfield type	Description
Subfield	Character	Character that separates subfields in a flat file document.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the subfield delimiter for this document is located. For instance, if you specify 5 as the character position, you have indicated that the subfield delimiter appears in the sixth character position from the beginning of the document.

Fixed length	Quoted release character type	Description
Quoted Release Character	Character	Character used to enable a section of text within a field to be represented as its literal value. Any delimiter characters that appear within this section will not be treated as delimiters. For instance, your field delimiter is (,) and your release character is “. When you want to use (,) within a field as text, you must prefix it with your quoted release character. When using the convertFlatFileToDocument operation to create the strings `Doe`, `John` and `Doe, Jane`, the record would appear as “`Doe, John”,“Doe, Jane`”. When using the convertDocumentToFlatFile operation to create “`Doe, John`”,“`Doe, Jane`”, the value of the record would be `Doe, John and Doe, Jane`. When using the convertDocumentToFlatFile operation, if you have specified both the Release Character and the Quoted Release Character, the Quoted Release Character is used.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the quoted release character for this document is located. For instance, if you specify 5 as the character position, you have indicated that the quoted release character appears in the sixth character position from the beginning of the document.

Fixed Length	Release Character type	Description
Release Character	Character	Character used to enable a delimiter to be used for its intended, original meaning. The character following the release character will not be treated as a delimiter.For instance, your field delimiter is + and your release character is \. When using + within a field as text, you must prefix it with your release character. When using the convertFlatFileToDocument operation to create the strings a+b+c and d+e+f, the record would appear as a\+b\+c+d\+e\+f. When using the convertDocumentToFlatFile operation to create a\+b\+c+d\+e\+f, the value of the record would be a+b+c and d+e+f.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the field delimiter for this document is located. For instance, if you specify 5 as the character position, you have indicated that the field delimiter appears in the sixth character position from the beginning of the document.

Next, set the record identifier.

The Variable Length record parser type expects each record to be preceded by two bytes that indicate the length of the record. Each record may be a different length.

To configure a variable length record parser, in the Parser area, select Variable length, and specify the following fields:

Variable Length	Field or Composite type	Description
Field or Composite	Character	Character that separates fields or composites in a flat file document.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero \(0\), the character position at which the field delimiter for this document is located. For instance, if you specify 4 as the character position, you have indicated that the field delimiter appears in the fifth character position from the beginning of the document.

Variable Length	Subfield type	Description
Subfield	Character	Character that separates subfields in a flat file document.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the subfield delimiter for this document is located. For instance, if you specify 5 as the character position, you have indicated that the subfield delimiter appears in the sixth character position from the beginning of the document.

Variable Length	Quoted Release Character type	Description
Quoted Release Character	Character	Character used to enable a section of text within a field to be represented as its literal value. Any delimiter characters that appear within this section will not be treated as delimiters. For instance, your field delimiter is (,) and your release character is “. When you want to use (,) within a field as text, you must prefix it with your quoted release character. When using the convertFlatFileToDocument operation to create the strings `Doe`, `John`, and `Doe, Jane`, the record would appear as “`Doe, John”,“Doe, Jane`”. When using the convertDocumentToFlatFile operation to create “`Doe, John`”,“`Doe, Jane`”, the value of the record would be `Doe, John and Doe, Jane`. When using the convertDocumentToFlatFile operation, if you have specified both the Release Character and the Quoted Release Character, the Quoted Release Character is used.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the quoted release character for this document is located. For instance, if you specify 5 as the character position, you have indicated that the quoted release character appears in the sixth character position from the beginning of the document.

Variable Length	Release Character type	Description
Release Character	Character	Character used to enable a delimiter to be used for its intended, original meaning. The character following the release character will not be treated as a delimiter. For instance, your field delimiter is + and your release character is \. When using + within a field as text, you must prefix it with your release character. When using the convertFlatFileToDocument operation to create the strings a+b+c and d+e+f, the record appears as a\+b\+c+d\+e\+f. When using the convertDocumentToFlatFile operation to create a\+b\+c+d\+e\+f, the value of the record appears as a+b+c and d+e+f.
	--OR--
	Character Position	Starting from the beginning of the document and counting from zero (0), the character position at which the field delimiter for this document is located. For instance, if you specify 5 as the character position, you have indicated that the field delimiter appears in the sixth character position from the beginning of the document.

Specify a Record Identifier.

When parsing a file, webMethods Integration looks at a record and extracts an identifier out of the data and uses that identifier to connect the record definition with a particular record in the flat file. The name of the record definition must match the value that is obtained by the record identifier.

To set the record identifier for a definition, if the flat file contains a record identifier, select Yes in the Record Identifier area, and set the record identifier to one of the following values.

Note: The Record Identifier Position has a default value that is selected as Start at field position.

Record Identifier	Value	Description
	Start at field position	Identifies the field in the record (counting from zero) that contains the identifier. Start at field Position identifiers use the value of the specified field as the record identifier. These identifiers count from zero (0).For instance, if 2 is specified, the third field is used as the record identifier.
	Start at the character position	Identifies the character position in the record (counting from zero) where the record identifier is located. Start at character position record identifiers compare the value that occurs in the record, at the specified offset, to all the record names defined in the flat file structure. The start at character position identifier cannot distinguish between all types of record names. For instance, if you name records “Rec1” and “Rec,” some instances of “Rec1” may be identified as “Rec,” because “Rec1” begins with “Rec.”

Click Next to define the Flat File Structure. Use the Flat File Structure page to add records to the flat file structure and define the hierarchical relationships between them.
The connector receiving the flat file uses the structure to read the flat file. This structural information identifies the parent-child relationships between different records in the flat file. By nesting record elements in the flat file structure (adding record elements to a record), you can represent the hierarchical structure of the data in the flat file.
Enable the Edit Mode in the Data fields panel on the Structure page to:
- add a new element using the "+" icon and select Record Definition from the Element type field to specify a name for the record definition. For more information, see the Usage Notes under step 11 of creating-a-flat-file-connector-from-a-sample-file.
- quickly navigate within the schema using the right, left, up and down arrows,
- perform edit actions For instance cut, copy, paste, and delete fields to modify the fields of the element in the property panel within the file schema, and
- utilize the keyboard shortcuts for modifying the flat file schema fields.
Note: The Show Data feature on the Data fields panel is available while creating the flat file using the import file function initially and not when you create the flat file manually or edit an existing flat file.

To add a Composite Definition after you have added a record definition, specify the following:

Extractor type	Description
Nth Field	Field number in the record that contains the composite you want to extract. This pulls the subfield data from the composite. If you leave this property empty, the composite will not be extracted.

To add a Field Definition after you have added a record definition, specify the following:

Extractor type	Description
Fixed Position	Counting from zero (0) indicates a fixed number of bytes to be extracted from a record.
	Position
Nth field	Counting from zero (0), indicates the field that you want to extract from the record.
	Extractor
	Type a value to indicate the position of the field that you want to extract from the record. This value cannot be null and must be an integer greater than or equal to zero (0). For instance, if you type 1, the second field is extracted.

This option is available only if you specified a field delimiter when configuring the definition and structure of the Flat File connector. This extractor returns the field as a key–value pair. The key is the name of the field. The value is the String value of the field.

Note: Make sure that the Record identifier position and Extractor type field values are the same for Field Definitions, else you might encounter errors while running the connector. For instance, if you have selected the Record identifier position as Start at character position, then the Extractor type must be set to Fixed position. If you have selected the Record identifier position as Start at field Position, then the Extractor type must be set to Nth position.

Click Next, the Summary page displays the Overview tab, which shows the details under the Description and the File Type and the Structure tab, which displays both the file schema and sample output.
Click Save to create the flat file connector. You are redirected to the Flat File Connectors page.

What to do next

You can edit or remove a flat file connector from the Flat File Connector page.