Topic
  • 3 replies
  • Latest Post - ‏2013-06-19T18:53:02Z by HermannSW
FrancoisDouglas
FrancoisDouglas
2 Posts

Pinned topic DataPower gzip decompression

‏2013-06-18T12:02:29Z |

Good day

I have a problem with a compressed message that I receive from a client and have not been able to resolve this, neither have a few resources I asked to help out.

I receive an XML message from a client that is Compressed(gzip), Encrypted and Signed using C#.

I am able to Verify the signature and decrypt the message successfully, the problem occurs once I try to decompress the message content.

What the client has done is to only compress the soap:Body content of the message and place it in a <CompressedBody> element.

Once I try to decompress this body content I end up with an empty string. The xpath is correct and working 100% as I have a message that I have compressed on DataPower itself using the gzip compression algorithm and decompressing this works great, however I am unable to decompress the client's data even when trying to use different compression algorithms.

The client has provided me with the C# code they use to compress the message. In this code it is specified that the message gets encoded with base-64 after the compression, however taking a look at the string that needs to be decompressed it contains the signs of a normally gzip compressed message:

H4sIAAAAAAAEAOy9B2AcSZYlJi9tynt/SvVK1+B0oQiAYBMk2J... ...D//4BeBeL4vwAA

Also when trying to decode this string with base-64 I receive the following error on DataPower:

Valid base64 passed to dp:decode resulted in non UTF8:

When I try to decode my compressed message - for debugging purposes - I receive the same error, which proves to me that this string is nor encoded.

Any help or suggestions would be much appreciated.

 

  • HermannSW
    HermannSW
    4725 Posts

    Re: DataPower gzip decompression

    ‏2013-06-18T15:52:49Z  

    Hi,

    DataPower internal encoding is UTF-8, see slide 11 of this WSTE webcast:
    http://www-01.ibm.com/support/docview.wss?uid=swg27022979

    Therefore any Non-UTF-8 result of "dp:decode(_, 'base-64')"  would result in problems and is not allowed.

    In fact you can make use of this feature to validate that a given binary input is in fact UTF-8 encoded text, see slide 16.

    > When I try to decode my compressed message - for debugging purposes - I receive the same error, ...
    >
    for debugging purposes you can send the data off box by <dp:url-open target="..." data-type="base64">yourBase64Data</dp:url-open>.

    If you do that, the data-type="base64" will base64-decode the data before sending.


    Now it depends what you want to do with the base64 data.
    There is a 2nd WSTE webcast on Non-XML data processing withing DataPower stylesheets on advanced topics:
    http://www-01.ibm.com/support/docview.wss?uid=swg27022979


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>
     

  • FrancoisDouglas
    FrancoisDouglas
    2 Posts

    Re: DataPower gzip decompression

    ‏2013-06-19T13:40:18Z  
    • HermannSW
    • ‏2013-06-18T15:52:49Z

    Hi,

    DataPower internal encoding is UTF-8, see slide 11 of this WSTE webcast:
    http://www-01.ibm.com/support/docview.wss?uid=swg27022979

    Therefore any Non-UTF-8 result of "dp:decode(_, 'base-64')"  would result in problems and is not allowed.

    In fact you can make use of this feature to validate that a given binary input is in fact UTF-8 encoded text, see slide 16.

    > When I try to decode my compressed message - for debugging purposes - I receive the same error, ...
    >
    for debugging purposes you can send the data off box by <dp:url-open target="..." data-type="base64">yourBase64Data</dp:url-open>.

    If you do that, the data-type="base64" will base64-decode the data before sending.


    Now it depends what you want to do with the base64 data.
    There is a 2nd WSTE webcast on Non-XML data processing withing DataPower stylesheets on advanced topics:
    http://www-01.ibm.com/support/docview.wss?uid=swg27022979


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>
     

    Hi HermannSW

     

    Thank you a lot for your reply on this and the great references.

    I have tried quite few ideas today. It appears that the client's message character encoding is ASCII inside UTF-8. This confuses me as I was under the impression that an XML message may be either or, not both.

    I have also tried to use your suggestion for debugging and using the <dp:url-open> to a difference service and I receive the following error:

    'illegal character 0x1f at line 1 of http://.../'

    This states that it has received an illegal XML character and hence I cannot continue trying this.

    I have tried to specify the output after capturing the the compressed body as ASCII and changed the XML declaration for the next transformation action to be ASCII as well to see if it made any difference, but it just continues as if I am using UTF-8.

    The reason I am trying to use it as ASCII is due to the fact that the compressed message does not seem to be UTF-8, so trying to wirk with it as ASCII might work.

    Do I have the wrong approach in how to change the character encoding to ASCII?

    Thank you once again for your response.

    Kind Regards

  • HermannSW
    HermannSW
    4725 Posts

    Re: DataPower gzip decompression

    ‏2013-06-19T18:53:02Z  

    Hi HermannSW

     

    Thank you a lot for your reply on this and the great references.

    I have tried quite few ideas today. It appears that the client's message character encoding is ASCII inside UTF-8. This confuses me as I was under the impression that an XML message may be either or, not both.

    I have also tried to use your suggestion for debugging and using the <dp:url-open> to a difference service and I receive the following error:

    'illegal character 0x1f at line 1 of http://.../'

    This states that it has received an illegal XML character and hence I cannot continue trying this.

    I have tried to specify the output after capturing the the compressed body as ASCII and changed the XML declaration for the next transformation action to be ASCII as well to see if it made any difference, but it just continues as if I am using UTF-8.

    The reason I am trying to use it as ASCII is due to the fact that the compressed message does not seem to be UTF-8, so trying to wirk with it as ASCII might work.

    Do I have the wrong approach in how to change the character encoding to ASCII?

    Thank you once again for your response.

    Kind Regards

    Hi,

    while 0x1f is an ASCII character, it is no XML Character.
    The only allowed XML control characters are tab, line feed and carriage return.

    So your input is Non-XML and cannot be processed by a stylesheet directly.

    See slide 7 of this WSTE webcast on how you can preprocess the Non-XML input to convert it to XML:http://www-01.ibm.com/support/docview.wss?uid=swg27019119
     

    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>