Topic
  • 8 replies
  • Latest Post - ‏2012-05-07T16:35:10Z by HermannSW
Stefan_Coach
Stefan_Coach
18 Posts

Pinned topic Datapower aborts when message contains Single quote

‏2012-05-02T12:16:27Z |
Hello,

We have an issue where an inbound message into Datapower is aborted during the parsing of the xml, it fails on the first node of our MPG because there is a "smart quote" within the body of the text. Basically if you enter a single quote in word ir gets replaced by this new quote and that is causing the issue. Has anyone experienced this before? Any suggestions on overcoming it?

Thanks

Stefan
Updated on 2012-05-07T16:35:10Z at 2012-05-07T16:35:10Z by HermannSW
  • UlrikAndersen
    UlrikAndersen
    262 Posts

    Re: Datapower aborts when message contains Single quote

    ‏2012-05-02T12:59:43Z  
    If you have "smart quotes" in the tags of the body of your message, that message is by definition not XML, and no XML parser can handle it.
    If you are using XI5x or XB6x, you could run the whole message through an ffd-based transform where you replace all "smart quotes" with normal quotes to make it XML.
    Or did I misunderstand the problem?
  • Stefan_Coach
    Stefan_Coach
    18 Posts

    Re: Datapower aborts when message contains Single quote

    ‏2012-05-03T18:16:50Z  
    Hi Ulrik,

    Actually the problem is that there are smart quotes in the body of the data, not in the xml tags themselves. So for example

    <location>Stefan's Office</location>

    The ' when a user copies and pastes from word ends up with the horrible microsoft smart quote and Datapower can't handle it currently. Our MPG is breaking before it even gets to the AAA step so I'm not sure how to intercept and resolve the issue.

    Stefan
  • HermannSW
    HermannSW
    7139 Posts

    Re: Datapower aborts when message contains Single quote

    ‏2012-05-03T23:38:29Z  
    Hi Ulrik,

    Actually the problem is that there are smart quotes in the body of the data, not in the xml tags themselves. So for example

    <location>Stefan's Office</location>

    The ' when a user copies and pastes from word ends up with the horrible microsoft smart quote and Datapower can't handle it currently. Our MPG is breaking before it even gets to the AAA step so I'm not sure how to intercept and resolve the issue.

    Stefan
    Hi Stefan,

    that sounds like your input is Non-XML and gets rejected.
    You may confirm that by looking into the log messages.

    If an input file has no XML declaration, then the encoding defaults to UTF-8 by the spec.
    If the actual encoding of the file is not UTF-8, then it may "look like" XML, but it is Non-XML.

    See "Repairing broken Web Services" on how you can deal with that if you cannot influence the client sending correct XML files:
    http://www-01.ibm.com/support/docview.wss?uid=swg27019119

     
    Hermann<myXsltBlog/> <myXsltTweets/>
  • Stefan_Coach
    Stefan_Coach
    18 Posts

    Re: Datapower aborts when message contains Single quote

    ‏2012-05-07T12:38:21Z  
    • HermannSW
    • ‏2012-05-03T23:38:29Z
    Hi Stefan,

    that sounds like your input is Non-XML and gets rejected.
    You may confirm that by looking into the log messages.

    If an input file has no XML declaration, then the encoding defaults to UTF-8 by the spec.
    If the actual encoding of the file is not UTF-8, then it may "look like" XML, but it is Non-XML.

    See "Repairing broken Web Services" on how you can deal with that if you cannot influence the client sending correct XML files:
    http://www-01.ibm.com/support/docview.wss?uid=swg27019119

     
    Hermann<myXsltBlog/> <myXsltTweets/>
    Hello All,

    Thanks for the feedback, however I believe the input is XML and furthermore a valid character. I do however think that this falls into a double byte character set. We are becoming more and more global and so the problem we are facing is more and more asian characters in our content that is being passed back and forth between systems.

    The "super quote" is a double byte character, and is just the beginning of the kinds of issues we will face if we don't have the ability to recieve this kind of information. I think this might just be a character set setting on Datapower, but I can't figure out which one.

    Our MPG fails on the very first node within the message, basically complaining that the XML is invalid. Although I have a long history of helpdesk, I wasn't intelligent enough to take a screen shot of the actual error.

    Stefan
  • Stefan_Coach
    Stefan_Coach
    18 Posts

    Re: Datapower aborts when message contains Single quote

    ‏2012-05-07T12:48:47Z  
    Hello All,

    Thanks for the feedback, however I believe the input is XML and furthermore a valid character. I do however think that this falls into a double byte character set. We are becoming more and more global and so the problem we are facing is more and more asian characters in our content that is being passed back and forth between systems.

    The "super quote" is a double byte character, and is just the beginning of the kinds of issues we will face if we don't have the ability to recieve this kind of information. I think this might just be a character set setting on Datapower, but I can't figure out which one.

    Our MPG fails on the very first node within the message, basically complaining that the XML is invalid. Although I have a long history of helpdesk, I wasn't intelligent enough to take a screen shot of the actual error.

    Stefan
    We reproduced the error in our Dev environment this morning to over come my inability to press print screen... Attached is the error we see when this "super quote" is in the message we are recieving.
  • HermannSW
    HermannSW
    7139 Posts

    Re: Datapower aborts when message contains Single quote

    ‏2012-05-07T13:30:10Z  
    We reproduced the error in our Dev environment this morning to over come my inability to press print screen... Attached is the error we see when this "super quote" is in the message we are recieving.
    Please attach (not paste) a sample failing XML input file here.

    Or, if this is not possible, please create a PMR so that I can get it from Level2 support that way.

     
    Hermann<myXsltBlog/> <myXsltTweets/>
  • Stefan_Coach
    Stefan_Coach
    18 Posts

    Re: Datapower aborts when message contains Single quote

    ‏2012-05-07T13:57:40Z  
    We reproduced the error in our Dev environment this morning to over come my inability to press print screen... Attached is the error we see when this "super quote" is in the message we are recieving.
    Hi,

    Attached is what is showing as sent content from the sending system.

    Stefan
  • HermannSW
    HermannSW
    7139 Posts

    Re: Datapower aborts when message contains Single quote

    ‏2012-05-07T16:35:10Z  
    Hi,

    Attached is what is showing as sent content from the sending system.

    Stefan
    Hi Stefan,

    from http://en.wikipedia.org/wiki/XML#Valid_characters:
    Unicode code points in the following ranges are valid in XML 1.0 documents:10

    • U+0009, U+000A, U+000D: these are the only C0 controls accepted in XML 1.0;
    • U+0020–U+D7FF, U+E000–U+FFFD: this excludes some (not all) non-characters in the BMP (all surrogates, U+FFFE and U+FFFF are forbidden);
    • U+10000–U+10FFFF: this includes all code points in supplementary planes, including non-characters.

    There are two characters with character code 25 (0x19) in your input making it Non-XML, find Non-DataPower proof below.

    Please either have the sending system send valid XML to DataPower,
    or use "Repairing broken Web services" from slide 6 of this WSTE webcast:
    http://www-01.ibm.com/support/docview.wss?uid=swg27019119



     
    Hermann<myXsltBlog/> <myXsltTweets/>