Topic
3 replies Latest Post - ‏2012-05-09T14:13:16Z by kimbert@uk.ibm.com
tharaniharan
tharaniharan
4 Posts
ACCEPTED ANSWER

Pinned topic MRM Parser - CVS, danish letters Æ Ø Å

‏2012-05-07T11:22:55Z |
Hello…
I’m new to IBM Message Broker, and currently working on a flow to read a csv file, and submit it to a queue.
So far I’m able to parse the csv file, map the values according to my messageset and add the data to a queue. The parsing performed by the MRM – CSV parser is successful when the file only contains US-letters. When I parse files containing Danish letters (æ,ø,å), it does not recognize them. The FileInput node does not throw an exception; instead it parses garbage for all occurrences of these three letters.

The ccsid for the file I’m parsing is: 1252 – I believe this is a common windows character endcoding.

I’ve parsed the file as a BLOB and in an ESQL node performed a CAST of the BLOB to CHARACTER, which gave me the hex value. I made a conversion using a hex-to-ASCII converter and here the letters æ,ø,å appears.

Could anyone please advice.
Updated on 2012-05-09T14:13:16Z at 2012-05-09T14:13:16Z by kimbert@uk.ibm.com
  • kimbert@uk.ibm.com
    kimbert@uk.ibm.com
    515 Posts
    ACCEPTED ANSWER

    Re: MRM Parser - CVS, danish letters Æ Ø Å

    ‏2012-05-08T11:49:38Z  in response to tharaniharan
    The csv file was written by another application. What encoding ( CCSID ) did that application use when it wrote the CSV file? You must use that CCSID in your message flow - no other CCSID will work correctly.
    • tharaniharan
      tharaniharan
      4 Posts
      ACCEPTED ANSWER

      Re: MRM Parser - CVS, danish letters Æ Ø Å

      ‏2012-05-09T13:30:18Z  in response to kimbert@uk.ibm.com
      I used firefox to determine the encoding, and found the matching CCSID. Worked like a charm :-). Thanks for you reply
  • kimbert@uk.ibm.com
    kimbert@uk.ibm.com
    515 Posts
    ACCEPTED ANSWER

    Re: MRM Parser - CVS, danish letters Æ Ø Å

    ‏2012-05-09T14:13:16Z  in response to tharaniharan
    How do you know that FireFox guessed correctly? FireFox did not talk to the app developers who wrote the CSV file - it just tried a few encodings on that particular example file until one of them "failed to produce an error".
    It is possible that CCSID 1252 is the correct choice. It is also possible that 1252 works for your example file but will not work for all files from that application. Some code pages are very similar to other code pages. I strongly recommend that you talk to the developers of the source application and find out what encoding they are using.