3 replies Latest Post - ‏2012-05-23T18:13:01Z by HermannSW
4372 Posts

Pinned topic A: yes -- Q: Is processing of Non-XML base64-string possible on XS40?

‏2011-02-21T13:27:12Z |
A colleague has a solution for his customer running fine on a XI50.

This solution gets a SAMLResponse base64-encoded string as Non-XML.
It uses binary wrapper technique to wrap the base64 string with an XML tag and then process further easily, see [1] or [2].

While this is fine for XI50 it only works with DataGlue license which is not available on XS40 devices.

The title already says that there is a solution --
not for the general problem of processing Non-XML data, but for dealing with a base64 string as input on XS40.

If you want to skip the explanation, just import attached backup and send a base64 string to port 3001 of your box,
see the very bottom that you will get then base64 data wrapped as XML for further processing.
The first idea I had was to utilize convert-http action, and in fact this is part of the solution.

So lets see what happens if this input file

$ cat t20.b64 AAA+ AA ++ $

is received by a Non-XML service with just a convert-http action (with "base64" default encoding):

$ curl --data-binary @t20.b64 http:
//cosmopolitan:3002; echo <request><url>/</url><base-url>/</base-url><args src=
"url"/><args src=
"body"><arg name=
"AAA  AA    " encoding=
"base64"> </arg> </args> </request> $

The whole base64 string is taken as "name" of 1st "name=value" pair for HTTP form.
The '+' signs get converted to spaces -- that is repairable.
But what is bad is that the newline chacters (&#10;) get converted to spaces due to attribute-value normalization, too:

$ curl -s --data-binary @t20.b64 http:
//cosmopolitan:3002 | od -tcx1 | head -14 | tail -4 0000120   g       n   a   m   e   =   
"   A   A   A           A   A     67 20 6e 61 6d 65 3d 22 41 41 41 20 20 41 41 20 0000140               
"       e   n   c   o   d   i   n   g   =   "   b 20 20 20 22 20 65 6e 63 6f 64 69 6e 67 3d 22 62 $

While for "normal" base64 strings newlines occur at specific places only (eg. by default "base64" command
wraps encoded lines after 76 columns) handling this is possible, but difficult at least.

So I remembered what my colleague said and that was the solution!
> ... Convert Input does not work, it's looking for name value pairs. ...

If we would receive "abc=base64string" everything would be fine.
We would get this:

<request><url>/</url><base-url>/</base-url><args src=
"url"/><args src=
"body"><arg name=
"abc" encoding=
"base64"> base64string </arg> </args> </request>

So now what we need is just a "prepend" service -- a service which is able to prepend the missing "abc=" to the input message.

From blog posting "Sending zip archives to DataPower":
> ...
> The binary input of Non-XML transformations needs to be "consumed", otherwise the default behavior is that it will be copied to the output.
> Attaching the input to the dummy SOAP file does not consume the input.
> ...

So the solution is to have two chained services.
The first MPGW service
  • is of Non-XML request type
  • matches all
  • fetches a file containing "prepend="
  • FETCH action is set to "binary" in advanced tab
  • backend is set to second XML FW service.

This MPGW just converts base64string to prepend=base64string and forwards to next service.

The second XML FW service
  • is of Non-XML request type
  • first action is convert-http with base64 default encoding
  • second transform action just takes the slightly modified base64 data, removes the newlines and replaces spaces by '+'

' &amp;#10;',


$ cat t22.b64 AAA+ AA == $ cat t21.b64 AAA+ AA += $ cat t20.b64 AAA+ AA ++ $ curl --data-binary @t22.b64 http:
//cosmopolitan:3001; echo <ret>AAA+AA==</ret> $ curl --data-binary @t21.b64 http:
//cosmopolitan:3001; echo <ret>AAA+AA+=</ret> $ curl --data-binary @t20.b64 http:
//cosmopolitan:3001; echo <ret>AAA+AA++</ret> $ $ cat b64.xsl <xsl:stylesheet version=
"1.0" xmlns:xsl=
"" xmlns:dp=
"" extension-element-prefixes=
"dp" > <xsl:output omit-xml-declaration=
"yes" /> <xsl:template match=
"/"> <ret> <!-- <xsl:value-of select=
"(/request/args/arg/@name)"/> --> <xsl:value-of select=
"translate(/request/args/arg,' &#10;','+')"/> </ret> </xsl:template> </xsl:stylesheet> $

Updated on 2012-05-23T18:13:01Z at 2012-05-23T18:13:01Z by HermannSW
  • HermannSW
    4372 Posts

    Re: A: yes -- Q: Is processing of Non-XML base64-string possible on XS40?

    ‏2011-02-21T13:32:18Z  in response to HermannSW
    The backup attached to previous posting was created with firmware.

    I just successfully upgraded the box to, deleted the domain and successfully imported the backup.

    If you want to look into the details without importing on a box have a look at attached backup documentation.