IBM Support

JR33566: USING ODBC ENTERPRISE TO READ COLUMNIZE UTF-8 SIMPLIFIED CHINESE DATA FROM DB2 BACK INTO DATASTAGE BUT ARE CORRUPTED.

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • Information Server 8.1 on AIX
    
    In DataStage, the customer is able to insert columnize UTF-8
    simplified Chinese data into a VARCHAR column in a UTF-8 DB2
    data base using an ODBC Enterprise stage.   Using the ODBC
    Enterprise stage attempt to read the data back into DataStage
    but are observing data corruption.  The customer has updated
    Data Direct drivers. Tried reading the data in as a VARCHAR and
    using a string_to_ustring conversion in a transfortmer without
    success.
    

Local fix

  • As a workaround the customer can read the data in as vargraphic
    e.g
    
    select ADMIN_SYS_TP_CD, CAST(ADMIN_CLIENT_ID as vargraphic(20))
    ADMIN_CLIENT_ID from IS_ADMINCLIENT
    
    In this case the ODBC stage should have ADMIN_CLIENT_ID as
    nvarchar .
    
    This fix is included in 8.0.1 fix pack 3
    

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    All platforms , jobs using ODBC EE to read UTF-8 characters
    ****************************************************************
    PROBLEM DESCRIPTION:
    Using the ODBC Enterprise stage to read columnize UTF-8
    simplified Chinese data from DB2 back into DataStage but are
    observing data truncation.
    ****************************************************************
    RECOMMENDATION:
    Install patch for JR33566
    The fix has been included in the IS 8.1 FP1. Apply the FixPack1
    for IS 8.1.
    ****************************************************************
    

Problem conclusion

  • When UTF-8 data is stored in char/varchar columns, ODBC EE is
    unable to read the data correctly as it fetches the data as
    SQL_C_CHAR type.
    
    Implemented option Use Ustrings, to read all char/varchar
    columns as unicode strings.
    When this option is selected , char/varchar columns are
    retrieved as SQL_C_WCHAR type.
    
    Note: Since we are attempting to read varchar/char columns as
    nvarchar/nchar , hence the output will be inflated
    nvarchar/nchar columns.
    
    e.g if the user is writing nvarchar(20) to a column of
    varchar(80), then it will be read back as nvarchar(80) . This
    is because the application cannot make a guess about the number
    of utf8 characters in the column. It could be from 20
    charactersto 80 characters.
      If the user changes the column definition in the GUI to
     nvarchar(20), they will see a warning message like
    
           "Implicit conversion from source type
    "ustring[max=80]" to     result type "ustring[max=20]":
    Possible truncation of variable  length string."
    
        To get the correct column
    definition, the customer should load  the imported table
    definition and check the box that says      "Ensure all char
    columns use unicode", whenever the option  Use UStrings is
    set to true.
    

Temporary fix

Comments

APAR Information

  • APAR number

    JR33566

  • Reported component name

    WIS DATASTAGE

  • Reported component ID

    5724Q36DS

  • Reported release

    801

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2009-07-06

  • Closed date

    2009-07-14

  • Last modified date

    2010-12-09

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Modules/Macros

  • ODBCEE
    

Fix information

  • Fixed component name

    WIS DATASTAGE

  • Fixed component ID

    5724Q36DS

Applicable component levels

  • R801 PSN

       UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSVSEF","label":"IBM InfoSphere DataStage"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.0.1","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
09 December 2010