Flat file connector

convertToString

Input parameters

ffValues

Document. The flat file document.

ffSchema

String. Namespace name of the flat file schema to use to convert the given document to a string.

spacePad

String. Optional. How to position the records in the flat file. Select from the following options:

• left: Left justify the records (add blank spaces to the right of the records) before the records are written to the output. This is the default.
• right: Right justify the records (add blank spaces to the left of the records) before the records are written to the output.
• none: No spaces added.

signalError

String. Whether to create errors in the output. Select from the following options:

• false: Do not create errors in output.
• true: Create errors in output.

noEmptyTrailingFields

String. Whether trailing empty fields are to be removed from the output. Used only with records that have delimited fields.

• true: Trailing empty fields are removed from the output. For example, if it is set to true, the output for a record with empty trailing fields looks like the following: AAA*01*02! (where ! is used as segment terminator). This is the default.
• false: A field separator remains to denote an empty field. For example, if it is set to false, the output for a record with empty trailing fields looks like the following: AAA*01*02********! (where ! is used as segment terminator).

noEmptyTrailingSubFields

String. Whether trailing empty subfields are to be removed from the output. Used only with records that have delimited fields. If no value is specified for the noEmptyTrailingSubFields parameter, Integration Server uses the value set for the noEmptyTrailingFields parameter.

• true: Trailing empty subfields are removed from the output.
• false: A field separator remains to denote an empty subfield.

delimiters

Document. Optional. The separator characters used to construct the output string. To specify a delimiter, you can specify:

• One character or character representation (for example, *, \n for line terminator, \t for tab)
• Hexadecimal value with prefix 0X (for example, 0X09, 0X13)
• Octal value with prefix 0 or decimal value (for example, 009, 013)
• Unicode characters (for example, \uXXXX where XXXX represents the Unicode value of the character)

Set the values for the following fields:

• record: String. Character to use to separate records. If you want to specify the two-character carriage return line feed (CRLF) characters, specify \r\n.
• field: String. Character to use to separate fields.
• subfield: String. Character to use to separate subfields.
• release: String. Character to use to ignore a record, field, or subfield delimiter in a field. If a release character occurs in a field or subfield before the delimiter, it is prefixed with release before being written to the output string.
• quotedRelease: String. Character to use to ignore a record, field, or subfield delimiter in a field. If a quoted release character occurs in a field or subfield before the delimiter, it is prefixed with quotedRelease before being written to the output string. The string is prefixed and appended with the quoted release character. For example, if * is a delimiter, the field value is a*b, and the quoted release character is ", the string appears as "a*b".
• lineSeparator: String. Optional. Character to use to override the input record delimiter when the following pre-condition is met and to handle the operating system specific newline. The record delimiter must be either "\r\n" [CRLF] or "\r" [CR] or "\n" [LF] and the parser is of Delimiter Type. Integration Server ignores lineSeparator if the above pre-condition is not met. Set the value for the following fields:
  • UNIX: Integration Server overrides the record delimiter with LF or "\n".
  • windows: Integration Server overrides the record delimiter with CRLF or "\r\n".
  • mac: Integration Server overrides the record delimiter with CR or "\r".
  • auto: Integration Server overrides the record delimiter with the operating system specific line separator.
• FormatInfo: Document. Any values mapped to the FormatInfo variable are passed unmodified to all format services invoked by convertToString and convertToValues.

encoding

String. The type of encoding used to write data to the output file. The default encoding is UTF-8. If the flat file document contains multi-byte encodings, you must install the Extended Character Set using the IBM webMethods Installer. For more information about installing the Extended Character Set, see the "Configuring the flat file byte count parser" section of the Flat File Schema Developer's Guide.

sortInput

String. Optional. Whether you want the service to sort the input records to match the flat file schema specified in ffSchema. Specify true for sortInput if the data in ffValues is not in the same order as defined by ffSchema.

• true: The service sorts the input records to match the flat file schema. If you select to sort the input records, note that:
  • The service runs slower.
  • All undefined records are sorted after the defined records.
  • The order of the undefined records that appear in the final document is random.
• false: The service does not sort the input records to match the flat file schema. The input records must match the order of the flat file schema. This is the default.

If there are multiple records at the same level with the same name, the order they appear in the final document is random.

returnAsBytes

String. Optional. Whether to return the output as a byte array instead of a string.

• false: Returns the document as a string. This is the default.
• true: Returns the document as a byte array instead of a string. This setting is useful when parsing multi-byte encodings.

Output parameter

string

String. Data that represents the flat file document.

bytes

Object. If the input variable returnAsBytes=true, returns the output as a byte array encoded using the specified encoding. The string value is not returned.

errorArray

Object. String array containing messages pertaining to errors that occurred during conversion. If no errors are encountered, this contains a value of null.

Usage Notes

When the convertToString service executes, the field that is defined to start after the end of the fixed length record will not be included in the output data if the following conditions are met:

• The flat file schema uses a fixed length record delimiter.
• The flat file schema contains a fixed position field that begins beyond the defined length of the fixed length record.
• The input to the convertToString service contains a value for the fixed position field that begins beyond the defined length of the fixed length record.

convertToValues

Converts flat file data into structured data by using the specified flat file schema.

Input parameter

ffData

Object. The flat file input with type of String, InputStream, or ByteArray.

ffSchema

String. The full name of the flat file schema object used to parse the ffData object.

ffIterator

Object. Optional. An object that encapsulates and keeps track of the input data during processing. It is used only when the iterate variable has been set to true.

encoding

String. Optional. The encoding of the InputStream passed in to ffData. The default encoding is UTF-8.

delimiters

Document. Optional. An document object that contains the segment terminator and the field and subfield separators. If the delimiter is null, it is located using the information defined in the flat file schema.

• One character or character representation (for example, *, \n for line terminator, \t for tab)
• Hexadecimal value with prefix 0X (for example, 0X09, 0X13)
• Octal value with prefix 0 or decimal value (for example, 011, 023)
• Unicode characters (for example, \uXXXX where XXXX represents the Unicode value of the character)
• The space character

Note: If you specify one delimiter value, you must specify all values. Specifying one of these values overrides any information set in the flat file schema.

• record (String) - Character used to separate records. To specify CRLF, use \r\n.
• field (String) - Character used to separate fields.
• subfield (String) - Character used to separate subfields.
• release (String) - Character used to ignore a record, field, or subfield delimiter in a field. If a release character occurs before the delimiter, it is prefixed before being written to ffValues.
• quotedRelease (String) - Character used to ignore delimiters in a field. If a quoted release character occurs before the delimiter, it is prefixed before being written to the output string. The string is prefixed and appended with the quoted release character.
  • Example: If * is a delimiter, the field value is a*b, and the quoted release character is ", the string appears as "a*b".
• FormatInfo (Document) - Any values mapped to FormatInfo are passed unmodified to all format services invoked by convertToString and convertToValues.

iterate

String. Optional. Whether you want to process the input all at one time.

batchsize

String. Optional. Whether you want to process the specified number of records. The input value of this field is a positive integer. This field is applicable only when the iterate field value is set to true. If you do not specify this field value, the default value is set to 1.

• true: Processes top-level records (children of the document root) in the flat file schema one at a time. After all child records of the top-level record are processed, the iterator moves to the top-level of the next record in the flat file schema, until all records are processed.
• false: Processes all input data at one time. This is the default.

createIfNull

String. Optional. Whether to create the document object if all the fields are null.

• true - No document (document object) is created if all fields are null. Default.
• false - Always create an document even if all fields are null.

skipWhiteSpace

String. Optional. Whether white space at the beginning of records is ignored.

Note: The fixed length record parser ignores skipWhiteSpace and preserves white space.

• true - Ignore leading white spaces in a record. Default.
• false - Record is used exactly as identified.

keepResults

String. Optional. Whether to return the parsed data in the ffValues output parameter.

• true - Parsed ffData is returned in ffValues. Default.
• false - ffValues does not return data. Useful when validating ffData against the schema.

validate

String. Optional. Whether to return error messages that describe how ffData differs from the flat file schema.

• true - Do not return validation error messages. Default.
• false - Return validation error messages.

returnErrors

String. Optional. Whether to return validation errors. Validation errors are returned only if validate=true.

• asArray - Return validation errors in the errors array. Default.
• inResults - Return validation errors in ffValues.
• both - Return validation errors in both errors and ffValues.

maxErrors

String. Optional. The maximum number of errors returned from one record. If more than the specified limit occur, parsing stops and returns data and errors collected up to that point.

flags

String. Optional. Flags that govern convertToValues options.

• addRecordCount: String. Whether you want the service to add an additional field (@record-count) to each parsed record in the resulting document object (ffValues). The @record-count field is used to identify the record number of each parsed record.
  • true - The @record-count field is added to each parsed record. This field contains the number of the parsed record. The first parsed record is 1, the second is 2, etc. If there are records that are undefined data, the count of the next defined record reflects the undefined data. For example, if the @record-count field for a record is 2 and that record contains 5 undefined records, the @record-count field for the next defined record is 8.
  • false - The @record-count field is not added to each parsed record. This is the default.
• detailedErrors: String. Whether you want detailed conditional validation error information. This flag is only used when validate is true.
  • true - When a conditional validation error occurs, the output errors variable contains detail information about all the conditions that were violated.
  • false - When a conditional validation error occurs, the service does not provide detail error information. Conditional validators report only whether a condition failed validation with no additional information about the conditions that were violated. This is the default.
• skipToFirstRecord: String. Whether you want the service to wait until it finds the first valid record before reporting invalid records as errors.
  • true - The service waits until it finds the first valid record before reporting invalid records as errors. This is the default.
  • false - The service reports invalid records as errors prior to locating the first valid record.
• trimWhitespace: String. Whether you want the service to delete any blank spaces at the beginning of fields, at the end of fields, or both.
  • none - The service does not delete any blank spaces from fields. This is the default.
  • left - The service deletes all blank spaces at the beginning of all fields.
  • right - The service deletes all blank spaces at the end of all fields.
  • both - The service deletes all blank spaces at the beginning and end of all fields.
• resultAsArray: String. Whether you want the service to return the ffValues output parameter as an document[] that can be mapped to the document types generated from the schema. An document[] is a document List. The resultAsArray parameter is used only when the iterate input parameter is set to true.
  • false - The service returns the ffValues output parameter as an document[] that can be mapped to the document types generated from the schema. This is the default.
  • true - The service returns the ffValues output parameter as an document object and not as an document[].

Output parameter

ffValues

Document. The document object that represents the input flat file data.

ffIterator

Object. Optional. An object that encapsulates and keeps track of the input records during processing. It is used only when the iterate variable has been set to true. When all input data has been processed, the object becomes null. When the ffIterator variable is null, exit the LOOP to discontinue processing.

isValid

String. Whether the flat file contains validation errors.

• true: The validate input variable was set to true and no errors were found.
• false: The validate input variable was set to true and errors were found, or the validate input variable was set to false.

errors

String. Optional. An array containing the validation errors, if any, that were found in ffData.

Usage notes

If you specified a default record definition by which the convertToValues service parses the document, the service displays the resulting recordWithNoID document as a child of the document above it, in an array.

To display the recordWithNoID record as a child of the root, change the value of the recWithNoIDLike46 to true in the Integration Server_directory \instances\instance_name\packages\WmFlatFile\config\ff file and reload the WmFlatFile package so that this configuration setting will take effect.

If some or all of the flat files that you process encounter problems because of memory constraints, set the iterator variable in the convertToValues service to true to process top-level records (children of the document root) in the flat file schema one at a time. After all child records of the top-level record are parsed, the convertToValues service returns and the iterator moves to the top-level of the next record in the schema, until all records are parsed. This parsing should be done in a flow service using a REPEAT step where each time the convertToValues service returns, the results are mapped and dropped from the pipeline to conserve memory. If the results were kept in the pipeline, out–of–memory errors might occur.