JR43624: SEQUENTIAL FILE STAGE REJECTS EXCESSIVE FIELDS IN A RECORD OF CSV FILES.

InfoSphere Information Server, Version 11.3 for Microsoft Windows
InfoSphere Information Server, Version 11.3 for AIX
InfoSphere Information Server, Version 11.3 for Linux
IBM InfoSphere Information Server Enterprise Edition V11.3 for Windows

APAR status

• Closed as fixed if next.

Error description

• The Enhancement:
  
    Sequential File Stage rejects excessive fields in a record of
  CSV files.
  
  The Details:
  
  ----------------------------------
  The properties of Sequential File Stage
  ----------------------------------
  - input/output
  -- columns
  
  1: column1, varchar(255),
  2: column2, number
  3: column3, varchar(255)
  
  -- formats
  --- Record level
  
  Final delimiter = end
  Record delimiter string = DOS format
  
  --- Field level
  
  Delimiter = comma
  Quotation = double
  
  ----------------------------
  A  CSV file
  ----------------------------
  "field 1", 0, "field 3"
  "field 1", 1
  "field 1", "2", "field 3", "fileld 4"
  ----------------------------
  
  The current result is as follows, dividing each record.
  
  |field 1|0|"field 3"|
  The second record is rejected since the number of column is
  short.
  |field 1|2|"field 3", "field 4"|
  
  The enhancement result is as follows dividing each record
  
  |field 1|0|field 3|
  The second record is rejected since the number of column is
  short.
  The third record is rejected since the number of column is
  excessive.
  
  Enhanced items:
  
      (1) When the number of columns in the table definition is 3,
  a record that has 3 or more comma is rejected.
      (2) On the current behavior, the final field regards commas
  and double quotations as general characters. On the enhanced
  behavior, the final field handles commas and double quotations
  as the same special characters with those of the first or the
  second field.
      (3) If this enhancement causes a compatible issue., an
  environmental value may be defined to switch both current and
  enhanced behaviors.
      (4) Any other stages that can define the table format should
  work similarly.
  
  
  Business merits:
  
      (1) The customer is free from bothering to handle the last
  field.
      (2) It can omit transformer stages to handle only the last
  field.
      (3) This enhancement reduces the number of stages in a job
  and make the Job stable and robust because the design becomes
  simpler.
      (4) The performance of DataStage improves and our sales
  teams become easier to sell DataStage to the customers.
  

Local fix

Problem summary

• When the number of fields in the incoming data doesn't match the
  number of fields defined in the stage schema, the record should
  be rejected if a user wants to reject the record.
  

Problem conclusion

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels

• R870 PSY

     UP