HermannSW

Con etiquetas: function

Determining the arity of XSLT functions

HermannSW Etiquetas: xquery xpath2 xslt function arity datapower 400 Visitas

Recently I got a stylesheet with really HUGE number of arguments in a concat() function.

My first approach of determining the arity was not nice and needed handwork, but did give the result I needed.

Later I learned from our XQuery team that XPath 2 has a much simpler solution for that:

Just replace concat(...) by count( (...) ) making the argument list an XPath 2 sequence and applying the count() function.

This approach is really easy, and can be evaluated by any XQuery processor.

With v6.0 firmware DataPower has an XQuery 1.0 compiler, so you can evaluate with DataPower.
You can also evaluate on any other XQuery 1.0 processor like saxon.
Or you can do it online at http://try.zorba.io.

So lets take this very shortened example here:

concat( ' ' , 'UNION ALL' ,' ', 'SELECT', ' ', 'XMLELEMENT', '(', $doubleQuotes, '...', $singleQuote, ',', '...')

It gets converted to this XPath 2 expression:

count( ( ' ' , 'UNION ALL' ,' ', 'SELECT', ' ', 'XMLELEMENT', '(', $doubleQuotes, '...', $singleQuote, ',', '...') )

Now only the definition of the used XSLT variables by dummies is needed:

declare variable $doubleQuotes := 'x';
declare variable $singleQuote := 'x';

So this is the whole thing:

declare variable $doubleQuotes := 'x';
declare variable $singleQuote := 'x';

count( ( ' ' , 'UNION ALL' ,' ', 'SELECT', ' ', 'XMLELEMENT', '(', $doubleQuotes, '...', $singleQuote, ',', '...') )

The result is "12", which you can see live evaluated on try.zorba.io (I do like the "Share" button):

http://try.zorba.io/#D6+GtvugeoNb5mK2GOHP80wcTMI=

Hermann_{<myXsltBlog/>} _{<myXsltTweets/>} _<myCE/>

Modificado por HermannSW

hex2num

HermannSW Etiquetas: stylesheet xslt extension function profiling datapower 926 Visitas

I do need a little helper function which converts a "byte" value represented by two hexadecimal digits into a number from 0-255.

Since this function will get called very often, choosing "the right" solution here is key.
This posting describes how DataPower stylesheet profoiling helped to identify the best alternative.

First I did come up with different solutions for solving the given task.

simply use dp:radix-convert($byte, 16, 10)
have a <xsl:choose> with 16 <xsl:when>s for hexdigit2num() and determine sum for both digits
improve 2. by combining cases '0'-'9' by test="$dig>=0" (only true for digits '0'-'9')

Next I just did put them into a stylesheet that exercises each function once.

Then I made the XML manager of coproc2 service to do stylesheet profiling by selecting a Profiling Rule in its Compile Options policy.

After flushing that XML manager ("profile", resets the counters) and sending 10000 requests I looked up "Status->XML Processing->Stylesheet Profiles".

Exercising the 10000 request was done this way:

$ for((f=1; f<=10000;++f))
> do
> echo "<x>AA</x>" | coproc2 hex.xsl - http://dp3-l3:2223
> done
170
...
170
$

Find the screenshots for inputs "00" , "99", "AA" and "FF" here (click to open big view):

This is just the table showing the measured times (on a 9004 XI50, reported in milli seconds , but with sub nano second precision):

	00	99	AA	FF	FF
#runs	10000	10000	10000	10000	20000
hex2num1	23	35	36	35	72
hexdig2num2	27	30	30	32	64
hex2num2	7	8	8	8	16
hexdig2num3	19	19	22	24	47
hex2num3	4	4	4	4	8

The increased values for *func:hexdig2num* are due to the fact that digit comparisons are done sequentially.
The last column gives just the values for doubled number of calls.
The worst case wrt time is input 'FF' which enforces quite some <xsl:when>s tests.

And the winner is "hex2num3 with hexdig2num3":
(4+24)/10000 = 2.8 [microsecond/call]

Here is the testing stylesheet:

Hermann_{<myXsltBlog/>} _{<myXsltTweets/>}

Doing recursion right

HermannSW Etiquetas: xslt smtp recursive datapower function 1.366 Visitas

In yesterday's developerWorks DataPower forum thread

http://www.ibm.com/developerworks/forums/thread.jspa?threadID=439588

customer complained that DataPower XSLT processor runs out of memory while processing a string of size 4MB:

wsgw (WS_HTTPS_SEND_MAIL): Execution of 'local:///dt01/sendMail.xsl' aborted: Out of memory

It turned out that he used functions from my smtp posting in this blog.

And it turns out that a simple helper function I provided (mime:split64()for splitting a long base64 string into lines of at most 76 characters length)

was not implemented according "Doing recursion right" technique mentioned in my 2010 WSTE webcast (slide 5).

Since DataPower XSLT processor does not provide tail-recursion optimization use of divide-and-conquer technique is required.
This recursive function runs out of stack space because a 4MB string results in more than 50.000 lines -- too much for recursion call-depth.

This is the devide-and-conquer technique which solves the problem easily -- determine $mid as offset where to split, "near" the middle (multiple of 76):

What can be learned from this?
Always implement "little helper functions" as efficient as possible -- you never know how others make use of them ... :-)

Hermann_{<myXsltBlog/>} _{<myXsltTweets/>}

DES "in" XSLT

HermannSW Etiquetas: xslt des extension function ecb xalan datapower cbc mode 2 Comentarios 1.819 Visitas

In a developerWorks DataPower Forum thread the question was raised on how to do ECB mode DES encryption with DataPower.

The answers were two-fold:

based on DataPower crypto extension function primitives DES is available in CBC mode only (cipher block chaining)
but that does not mean that you cannot do DES encryption in ECB mode:
just implement the cryptographic primitive (64bit block en-/de-crypt) in XSLT(!)

Find the implementation with a sample from "The DES Algorithm Illustrated" in action, portable for DataPower and xalan use:

https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14700744#14700744

A 64bit des:encrypt-blk() call takes roughly 3ms, which means that you do not want to apply that on big data.

The implementation works of 01-strings, hexTObin and other conversion functions included.

This reminds me 5.5 years back when I worked in Smartcard development department in Boeblingen Lab.

The person responsible for side channel attacks left IBM and I was the first who raised the hand for the equipment:

two oscilloscope cards for the server (100MHz, 2GHz)
a special card reader with high precision probe
many smartcards, some secure, others less secure, for adjustment
software for doing side channel attacks.

What was easy to do was to break triple DES on cards without randomization counter-measure based on statistical analysis of several thousand measurements.

What I heard about is the single request break of RSA private keys in early smardcards.
The code for doing exponentiation in that cards was "efficient", it did compute

x^(2*n) by (x^n)^2 and
x^(2*n+1) by ((x^n)^2)*x

So the single bits of the private key could be directly seen on the oscilloscope by long (odd exponent) or short (even exponent) areas of high power consumtion.

We did not have the equipment to cut reader power based on some runtime condition or at a specific clock cycle of computation.

But the certification agencies (and real attackers) had.

Once we got a complaint from a cert agency that a specific pattern on the oscilloscope would leak information.

They claimed that the pattern corresponds to a specific part of the OS code.

Using the oscilloscope I was able to disprove that statement.

I just added code that created spikes at specific locations in the OS code.

Then running the request the oscilloscope proved that the pattern of the complaint belonged to a totally different part of the OS code.
Now how to generate a "spike"?
That was easy -- just power on the (hardware) random generator and immediately power it off with the next command.

But that was long ago, before IBM sold its smartcard business and I joined DataPower development 4.5 years ago.

Hermann.

Decoding quoted-printable string inside a stylesheet

HermannSW Etiquetas: recursive datapower xslt function 1.046 Visitas

The last posting showed how to process quoted-printable encoded root section of a multipart/signed message by a simple transformation with service chaining.

If you
* need to decode a quote-printable string

* do not like service chaining
* do not like to have a seperate WTX binary transform action with QUOTEDTOTEXT() function

then the single stylesheet solution posted here may be what you want to try out:

http://www.ibm.com/developerworks/forums/thread.jspa?threadID=391105

Attached to that posting is a correctly formatted multipart/signed sample message (with CRLFs) and the stylesheet.

This sytelsheet is a nice demonstration of these features:
* accessing Non-XML input

* UTF-8 validate input (needed for safety, OK for scenario since ASCII is a subset of UTF-8)
* having a func:function doing some preparation work (eliminate soft line breaks, split at '=3D')
* having a recursive func:function doing the rest.

Important for the recursive function is doing only what needs to be done in the stylesheet and rely on efficient extension functions (regexp:replace) where possible.

Last, but not least, it demonstartes how you can map a unicode code point to a parsed charecter entity inside a stylesheet by mapping (for ISO-8859-1 sample).

Hermann.

xs:decimal()

HermannSW Etiquetas: coproc2 function xslt recursive sum datapower xs:decimal 2.0 1.783 Visitas

XSLT 2.0 data type xs:decimal is available on DataPower:
http://publib.boulder.ibm.com/infocenter/wsdatap/v3r8m1/index.jsp?topic=/xs40/extensionfunctions146.htm

By the XPath 2.0 spec xs:decimal has to be applied to an atomic type:
http://www.w3.org/TR/xpath-functions/#constructor-functions-for-xsd-types
...
* xs:decimal($arg as xs:anyAtomicType?) as xs:decimal?
...

If you need to apply xs:decimal() to a node-set you may want to use recursive function func:sum() from this stylesheet martins.xsl:

<xsl:stylesheet version="2.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  xmlns:func="http://exslt.org/functions"
>
  <xsl:output method="text"/>
 
  <func:function name="func:sum">
    <xsl:param name="nodeset"/>
    <xsl:choose>
      <xsl:when test="count($nodeset)=1">
        <func:result select="xs:decimal($nodeset[1])"/>
      </xsl:when>
      <xsl:otherwise>
        <xsl:variable name="mid" select="floor(count($nodeset) div 2)"/>
        <func:result select="xs:decimal(func:sum($nodeset[position() &lt;= $mid])) +
                             xs:decimal(func:sum($nodeset[position() > $mid]))"/>
      </xsl:otherwise>
    </xsl:choose>
  </func:function>
 
  <xsl:template match="/">
    <xsl:variable name="netValue" select="'69.50'"/>
    <xsl:text>
</xsl:text>
    <xsl:variable name="orderCondNode">
      <CondValue>9.04</CondValue>
      <CondValue>1.00</CondValue>
      <CondValue>1.00</CondValue>
      <CondValue>1.00</CondValue>
    </xsl:variable>
 
    <xsl:variable name="sum">
      <xsl:value-of select="xs:decimal($netValue) + func:sum($orderCondNode/CondValue)"/> 
    </xsl:variable>
 
    <expected>81.54</expected>
    <xsl:text>
</xsl:text>
 
    <sum><xsl:value-of select="$sum"/></sum>
    <xsl:text>
</xsl:text>
  </xsl:template> 

</xsl:stylesheet>

Be aware that xs:decimal does not work without "XSLT 2.0" being selected in Compile options policy of the XML manager:


$ java coproc2 martins.xsl some.xml http://dp5-l3.boeblingen.de.ibm.com:2223

81.54
NaN
$

With "XSLT 2.0" being selected in Compile options policy of the XML manager (here for coproc2 service, or for your service) everything is fine:

$ java coproc2 martins.xsl some.xml http://dp3-l3.boeblingen.de.ibm.com:2223

81.54
81.54
$

Hermann.

Canal de información para entradas de blogs ▼

HermannSW

▼ Etiquetas

▼ Blogs parecidos

WebSphere and...

Asset Managem...

developerWork...

AIX Down Unde...

Hazem Ahmed S...

▼ Blogs de ideación parecidos

IBM Forms Exp...

Rational Soft...

▼ Archivador

▼ Autores de blog

HermannSW

Determining the arity of XSLT functions

hex2num

Doing recursion right

DES "in" XSLT

Decoding quoted-printable string inside a stylesheet

xs:decimal()