Unusual behaviors with JSON-string parameter in SOAP response from server through Datapower X18.104.22.168.1 version.
We are working in environment with an HTTPS SOAP web service that responds to requests to executeWithString() method by returning a single string (that happens to be a JSON-encoded string within the usual SOAP XML response structure). Our logs indicate that the returned JSON string is “normal”; however, under a few unusual situations that string ends up corrupted when it reaches the Java runtime client – passing through a Datapower appliance.
(1) Periodically (about 1% of responses), there occurs a sub-string pattern in the response at runtime server of the following
at one of the following byte positions in the string: 5806, 7854, 9902 – (note interval of 2048 bytes between positions). When received at runtime client, an extra character has been inserted – and the client naturally fails its JSON parsing:
Total string size of the response varies from around 5,000 to nearly 20,000 bytes in size.
(2) Once, where this pattern occurred at position 7854, the received string at the client was missing a 2048 byte chunk.
I am very fuzzy about what does or what could happen with the Datapower box that might somehow modify the response string? I saw some reports (APAR http://www-01.ibm.com/support/docview.wss?uid=swg1IC75163); but the description of the behavior does not match our observed behavior. AND, really, should this returned string parameter simply be treated as a string with respect to SOAP and Datapower? Unless some transformation policy was in effect?
At this point, any thoughts, suggestions, etc. would be appreciated. Our team does NOT have ready access to the Datapower box UI, so we need to sleuth and infer through the internet primarily.
I have uploaded the entire SOAP XML string passed back from the webservice. You’ll note that all Unicode characters are now ascii-encoded.
best regards & thanks,
Pinned topic JSON string parameter in SOAP response occasionally gets corrupted
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
Updated on 2013-04-04T15:09:57Z at 2013-04-04T15:09:57Z by SystemAdmin
HermannSW 2700006U547225 Posts
Re: JSON string parameter in SOAP response occasionally gets corrupted2013-02-20T23:03:08ZThis is the accepted answer. This is the accepted answer.You are really backlevel with 22.214.171.124 firmware, latest 3.8.2 firmware is 126.96.36.199:
There are a lot of APARs that have been fixed after 188.8.131.52.
In case you do have a binary Transform action you might have hit APAR IC81253 fixed in 184.108.40.206.
(that bug changed string values)
You should upgrade one of your boxes to 220.127.116.11 and retest.
If the issue is gone, it has already been fixed.
If not, please create a PMR.
Re: JSON string parameter in SOAP response occasionally gets corrupted2013-03-15T14:22:54ZThis is the accepted answer. This is the accepted answer.
- HermannSW 2700006U54
The firmware has been upgraded a couple of months ago (to v 18.104.22.168). It is likely that the string corruption problem may have evolved with that upgrade from Inserting an extra Double-quote to dropping 2048 bytes of response.
When viewing the SOAP response with JSON parameter "inside" of Datapower via web UI, the corruption may not be visible (see attached); however, the Java Runtime Client receives corrupted string.
When same request is tested WITHOUT DP in picture, there is no corruption (see network picture also attached). The string is normal when viewed (in green arrow) but is corrupted (orange) when received at JAVA RTC (*only* if DP in network).
Note that sample non-corrupted SOAP message attached in original message.
(1) Is there anyway to precisely examine the SOAP response string stream OUT of DP with DP diagnostics (or do we have to use network sniffing tool)?
(2) Logically, I cannot see what else can be immediate cause of corruption besides Datapower? I have spent long hours combing IBM Support and do see various kinds of bugs - but not exactly matching these conditions.
Is it reasonable to assume that there is bug in DP?
(3) Any other advice (please) for filing a PMR; for example, what additional details are required??
Many thanks for any assistance or guidance,
HermannSW 2700006U547225 Posts
Re: JSON string parameter in SOAP response occasionally gets corrupted2013-03-15T22:25:55ZThis is the accepted answer. This is the accepted answer.
- SystemAdmin 110000D4XK
the APAR I mentioned was fixed on 22.214.171.124 and 126.96.36.199.
Please upgrade to 188.8.131.52 and retest.
> In case you do have a binary Transform action you might have hit APAR IC81253 fixed in 184.108.40.206.
> (that bug changed string values)
Or even better, upgrade to 220.127.116.11 for testing.
If it still fails, just open a PMR.
Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>
Re: JSON string parameter in SOAP response occasionally gets corrupted2013-04-04T15:09:57ZThis is the accepted answer. This is the accepted answer.
- HermannSW 2700006U54
Thank you - I appreciate your assistance. We are slowly pursuing this with our client and IBM Thailand, trying to reproduce the corruption bug using SOAPUI to simulate server. Meanwhile, I have uploaded a slightly revised (better capture) of the SOAP message that gets corrupted. I realized that the original text submitted had some whitespace deleted. I also noticed (just a little striking) that opening the entire SOAP message text file with a binary editor, the critical pattern falls a few bytes after 2000 hex position.
Thanks again for your help,