PK89778: DBCS CHAR WITH JIS RESULTS IN BAD OUTPUT

APAR status

• Closed as program error.

Error description

• When using JIS code for Japanese text, and the string is one
  character, JIS escape code 0x1B2842 will be broken. (It seems
  that the escape code and delimiter "," (0x2C) were mixed.)
  
  This map converts XML data to delimited flat file (CSV). When
  using JIS code for Japanese text, and the string is one
  character, JIS escape code 0x1B2842 will be broken. (It seems
  that the escape code and delimiter "," (0x2C) were mixed.)
  
  Following is hex dump of output.txt
  ----------
  43 75 73 74 6F 6D 65 72 2C 30 30 30 31 2C "1B 24
  42" 42 67 "1B 2C 28 42" 1B 24 42 45 44 43 66 1B 28 42
  ^^^^^^^^^^^^^^^^^^^^^^^
  ----------
  "2C" on 0x00000D is delimiter, and next "1B2442" is JIS escape
  code, and next "4267" is Japanese character sequence. Next
  "1B2C2842" should be "1B2842" + "2C" (JIS escape + delimiter),
  but these were mixed.  This mixed code will be displayed as
  garbage.
  
  This problem also occurs when using "ISO 2022,
  locale=ja,version=0" or other JIS codesets instead of "JIS
  (Depricated)".
  
  The problem does not occur when the JIS string has more than two
  Japanese characters or using SJIS or other Japanese codeset.
  
  WTX VERSION: WTX8.2.0.4 IFix1
  
  PLATFORM: WindowsXP SP2
  

Local fix

• WTXCQ - WTX00030927
  BD  / BD
  Changes in the code which requires a patch.
  Circumvention: None
  

Problem summary

• Users Affected:
  All Websphere Transformation Extender (WTX) customers who run
  maps with JIS code page using Command Server, Design Studio and
  Launcher.
  
  Problem Description:
  The issue has been resolved with the Core Engine. Now, ICU UTF-8
  to JIS (Deprecated) would convert the Japanese UTF-8 character
  to JIS(Deprecated) character. The buffer overflow during the
  conversion of ICU UTF-8 japanese character to JIS (Deprecated)
  character resulted in this behavior. Japanese characters would
  now work fine with the mapping.
  Platforms Affected:
  All
  

Problem conclusion

• Fixed a problem with Engine where it does not convert UTF-8
  Japanese character to JIS character properly due to Buffer
  overlow with ICU conversion function.
  
  Delivered in:
  8.1.0.5, 8.2.0.5, 8.3.0.0
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WSTX

• Fixed component ID

  5724M4000

Applicable component levels

• R810 PSY

     UP

• R820 PSY

     UP