Topic
IC4NOTICE: developerWorks Community will be offline May 29-30, 2015 while we upgrade to the latest version of IBM Connections. For more information, read our upgrade FAQ.
4 replies Latest Post - ‏2013-04-04T15:09:57Z by SystemAdmin
SystemAdmin
SystemAdmin
6772 Posts
ACCEPTED ANSWER

Pinned topic JSON string parameter in SOAP response occasionally gets corrupted

‏2013-02-20T20:45:20Z |
Unusual behaviors with JSON-string parameter in SOAP response from server through Datapower X150.3.8.2.1 version.

We are working in environment with an HTTPS SOAP web service that responds to requests to executeWithString() method by returning a single string (that happens to be a JSON-encoded string within the usual SOAP XML response structure). Our logs indicate that the returned JSON string is “normal”; however, under a few unusual situations that string ends up corrupted when it reaches the Java runtime client – passing through a Datapower appliance.

(1) Periodically (about 1% of responses), there occurs a sub-string pattern in the response at runtime server of the following

"}],

at one of the following byte positions in the string: 5806, 7854, 9902 – (note interval of 2048 bytes between positions). When received at runtime client, an extra character has been inserted – and the client naturally fails its JSON parsing:

""}],

Total string size of the response varies from around 5,000 to nearly 20,000 bytes in size.
(2) Once, where this pattern occurred at position 7854, the received string at the client was missing a 2048 byte chunk.
I am very fuzzy about what does or what could happen with the Datapower box that might somehow modify the response string? I saw some reports (APAR http://www-01.ibm.com/support/docview.wss?uid=swg1IC75163); but the description of the behavior does not match our observed behavior. AND, really, should this returned string parameter simply be treated as a string with respect to SOAP and Datapower? Unless some transformation policy was in effect?

At this point, any thoughts, suggestions, etc. would be appreciated. Our team does NOT have ready access to the Datapower box UI, so we need to sleuth and infer through the internet primarily.

I have uploaded the entire SOAP XML string passed back from the webservice. You’ll note that all Unicode characters are now ascii-encoded.

best regards & thanks,
michael
Updated on 2013-04-04T15:09:57Z at 2013-04-04T15:09:57Z by SystemAdmin
  • HermannSW
    HermannSW
    4490 Posts
    ACCEPTED ANSWER

    Re: JSON string parameter in SOAP response occasionally gets corrupted

    ‏2013-02-20T23:03:08Z  in response to SystemAdmin
    You are really backlevel with 3.8.2.1 firmware, latest 3.8.2 firmware is 3.8.2.17:
    http://www-01.ibm.com/support/docview.wss?uid=swg24028338#3.8.2.17

    There are a lot of APARs that have been fixed after 3.8.2.1.
    In case you do have a binary Transform action you might have hit APAR IC81253 fixed in 3.8.2.14.
    (that bug changed string values)

    You should upgrade one of your boxes to 3.8.2.17 and retest.
    If the issue is gone, it has already been fixed.
    If not, please create a PMR.

     
    Hermann<myXsltBlog/> <myXsltTweets/>
    • SystemAdmin
      SystemAdmin
      6772 Posts
      ACCEPTED ANSWER

      Re: JSON string parameter in SOAP response occasionally gets corrupted

      ‏2013-03-15T14:22:54Z  in response to HermannSW
      Followup.

      The firmware has been upgraded a couple of months ago (to v 5.0.0.0). It is likely that the string corruption problem may have evolved with that upgrade from Inserting an extra Double-quote to dropping 2048 bytes of response.

      When viewing the SOAP response with JSON parameter "inside" of Datapower via web UI, the corruption may not be visible (see attached); however, the Java Runtime Client receives corrupted string.

      When same request is tested WITHOUT DP in picture, there is no corruption (see network picture also attached). The string is normal when viewed (in green arrow) but is corrupted (orange) when received at JAVA RTC (*only* if DP in network).

      Note that sample non-corrupted SOAP message attached in original message.

      Questions:

      (1) Is there anyway to precisely examine the SOAP response string stream OUT of DP with DP diagnostics (or do we have to use network sniffing tool)?

      (2) Logically, I cannot see what else can be immediate cause of corruption besides Datapower? I have spent long hours combing IBM Support and do see various kinds of bugs - but not exactly matching these conditions.

      Is it reasonable to assume that there is bug in DP?

      (3) Any other advice (please) for filing a PMR; for example, what additional details are required??

      Many thanks for any assistance or guidance,

      Michael
      • HermannSW
        HermannSW
        4490 Posts
        ACCEPTED ANSWER

        Re: JSON string parameter in SOAP response occasionally gets corrupted

        ‏2013-03-15T22:25:55Z  in response to SystemAdmin
        Hi Michael,

        the APAR I mentioned was fixed on 3.8.2.14 and 5.0.0.1.
        Please upgrade to 5.0.0.1 and retest.

        > In case you do have a binary Transform action you might have hit APAR IC81253 fixed in 3.8.2.14.
        > (that bug changed string values)

        Or even better, upgrade to 5.0.0.5 for testing.
        If it still fails, just open a PMR.

         
        Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>
        • SystemAdmin
          SystemAdmin
          6772 Posts
          ACCEPTED ANSWER

          Re: JSON string parameter in SOAP response occasionally gets corrupted

          ‏2013-04-04T15:09:57Z  in response to HermannSW
          Hello Hermann,

          Thank you - I appreciate your assistance. We are slowly pursuing this with our client and IBM Thailand, trying to reproduce the corruption bug using SOAPUI to simulate server. Meanwhile, I have uploaded a slightly revised (better capture) of the SOAP message that gets corrupted. I realized that the original text submitted had some whitespace deleted. I also noticed (just a little striking) that opening the entire SOAP message text file with a binary editor, the critical pattern falls a few bytes after 2000 hex position.

          Thanks again for your help,
          Michael