Topic
  • 18 replies
  • Latest Post - ‏2013-02-20T12:07:20Z by HermannSW
SystemAdmin
SystemAdmin
6772 Posts

Pinned topic coproc2 v1.3 -- version to upload multiple files

‏2012-09-18T08:04:05Z |
Hi,

I have found the coproc2 utilities to be very useful and use them often when working on xslt stylesheets for the datapower devices. I have recently been doing quite a bit of such work involving ffd files for input and output mapping and with user written xslt functions that i include in my xsl, and i was finding it a bit of a hassle when one or more of those files was changing -- each time i had to upload the changed files before submitting my coproc2 test and this seemed somewhat to defeat the convenience and utility of coproc2 in the first place. I was using the java coproc2 client, and so I made some small modifications to HermannSW's java coproc2 program v1.2, such that the modified program would as well take a list of file names (on the pc) and upload them as part of a new http header added to the coproc2 HTTP POST, and then as well I modified the coproc2.xsl so as to save these files to the temporary:/// directory of the coproc2 domain.

The coproc2.java, the jar for this, and the coproc2.xsl files are included in the attached zip.

If you want to try this, either compile the new version of coproc2.java or add the attached jar file to your classpath. Also, copy the new version of coproc2.xsl to local:// directory for the domain where you have installed the coproc2 utility services (coproc2 and coproc2nonxml).

The new version java client takes as paramaters:

* @version 1.3
* @param stylesheet2Send stylesheet to be executed on DataPower
* @param xmlFile2Send xmlfile sent to DataPower (or '-' for standard input)
* @param url URL of coproc2 Endpoint on DataPower
* @param filesToUpload an optional list of one or more semicolon separated filenames to be uploaded to
* "temporary:///" directory in coproc3 service domain. For example: myFile1;myFile2
* @param -v optional, do verbose output

The optional parameters can be in either order.

The modifications are backwardly compatible so you can continue to use java coproc2 client as previously, or if you want to upload some files along with the POST then use coproc2 with the optional filesToUpload parameter for example:

coproc2 myxslfile.xsl myxmlfile.xml http://mydp:2223 myFile1;myFile2:myFile3

In this case the 3 files (myFile1 myFile2 and myFile3) will be written to the temporary:// directory of the domain where your coproc2 utilities are installed prior to the invocation of the main transformation, that is, prior to the execution of the myxslfile.xsl stylesheet with myxmlfile.xml as input.

A minor hassle with this implementation is that since I used <dp:dump-nodes> as part of how to get the files uploaded, then they can only be written to the temporary:/// directory of the domain where your coproc2 services are installed, so what this means is that anywhere you refer to them in your main xsl file, you need to reference them in the temporary directory. For example, if i had an <dp:input-mapping> element, in my main xsl it might look like:


<dp:input-mapping      href=
"temporary:///utf-8.ffd" type=
"ffd"/>


or for functions or other included files:


<xsl:include href=
"temporary:///my_funcs.xsl" />

A brief explanation of the file upload implementation:

In the java coproc2 client, a serialized xml string with the file names and content is created and then passed as the value of a new HTTP header named "filesXMLString" included with the POST -- the filesToUpload parameter is parsed for file names and then for each file the name and contents are each base64 encoded and added as elements to the xml string.

In the coproc service, the string value for the filesXMLString is parsed, and then processed to write each file to the temporary directory using <dp:dump-nodes> element.
The code for the coproc2.xsl follows:


<xsl:stylesheet version=
"1.0" xmlns:xsl=
"http://www.w3.org/1999/XSL/Transform" xmlns:dp=
"http://www.datapower.com/extensions" xmlns:exsl=
"http://exslt.org/common" extension-element-prefixes=
"dp" > <xsl:output method=
"xml" omit-xml-declaration=
"yes"/> <xsl:template match=
"/"> <!-- store xsl in context variable (

for use in next transform action) --> <dp:set-variable name=
"'var://context/xsl'" value=
"dp:inflate(dp:http-request-header('xsl'),'gzip')"/>   <!-- force the skipping of backside send 

for 

this service --> <dp:set-variable name=
"'var://service/mpgw/skip-backside'" value=
"1"/>   <!-- write any uploaded files to the temporary directory --> <xsl:variable name=
"filesXMLString" select=
"dp:http-request-header('filesXMLString')"/> <xsl:

if test=
"$filesXMLString != ''"> <xsl:variable name=
"fileNode" select=
"dp:parse($filesXMLString)"/> <xsl:for-each select=
"$fileNode/files/file"> <dp:dump-nodes file=
"dp:inflate(./filename/text(),'gzip')" nodes=
"dp:parse(dp:inflate(./filecontent/text(),'gzip'))"/> </xsl:for-each> </xsl:if> </xsl:template> </xsl:stylesheet>

(One thing to note is that if you have an error in one of the files that you are uploading then this will often surface as a failed parse of some part of the filesXMLString -- check the default-log for the domain where you have installed the coproc2 services).
If you do try this and have any problems please do let me know. Also, any comments or suggestions are welcomed. Please let me know what you think.
craig oddy
Updated on 2013-02-20T12:07:20Z at 2013-02-20T12:07:20Z by HermannSW
  • HermannSW
    HermannSW
    4894 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2012-09-19T02:02:06Z  
    Hi Craig,

    really nice work!

    I added a reference to this thread in old coproc2 thread:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14888251#14888251

    You are right with the restriction to "temporary:" folder for <dp:dump-nodes ...>.
    If you want more flexibility and can have the files available on some HTTP server on your local machine boot.xsl technique might be interesting to you.

    I just tested your new coproc2 Java client with stylesheet T1.xsl and range.xsl from this posting:
    http://www.biglist.com/lists/lists.mulberrytech.com/xsl-list/archives/200907/msg00263.html

    The only change was to modify the xsl:include to point into "temporary:" folder.
    T1.xsl outputs the numbers from 1 to 10000 as a single string.
    Because the output is a little big "wc --bytes" just tells the length of the output string:
    
    $ java coproc2 T1.xsl empty.xml http:
    //dp9-l3:2223 range.xsl | wc --bytes 38895 $ $ grep include T1.xsl <xsl:include href=
    "temporary:///range.xsl"/> $
    


     
    Hermann<myXsltBlog/> <myXsltTweets/>
  • HermannSW
    HermannSW
    4894 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2012-09-19T02:20:27Z  
    • HermannSW
    • ‏2012-09-19T02:02:06Z
    Hi Craig,

    really nice work!

    I added a reference to this thread in old coproc2 thread:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14888251#14888251

    You are right with the restriction to "temporary:" folder for <dp:dump-nodes ...>.
    If you want more flexibility and can have the files available on some HTTP server on your local machine boot.xsl technique might be interesting to you.

    I just tested your new coproc2 Java client with stylesheet T1.xsl and range.xsl from this posting:
    http://www.biglist.com/lists/lists.mulberrytech.com/xsl-list/archives/200907/msg00263.html

    The only change was to modify the xsl:include to point into "temporary:" folder.
    T1.xsl outputs the numbers from 1 to 10000 as a single string.
    Because the output is a little big "wc --bytes" just tells the length of the output string:
    <pre class="jive-pre"> $ java coproc2 T1.xsl empty.xml http: //dp9-l3:2223 range.xsl | wc --bytes 38895 $ $ grep include T1.xsl <xsl:include href= "temporary:///range.xsl"/> $ </pre>

     
    Hermann<myXsltBlog/> <myXsltTweets/>
    Forgot to mention that coproc2 is fine for pre v5.0 DataPower firmware.

    Starting with DataPower 5.0.0.x firmware "Interoperability Test Service" allows to do much more than coproc2 can,
    out-of-the box with client on ressource kit, see:
    http://pic.dhe.ibm.com/infocenter/wsdatap/v5r0m0/nav/2_1_0

    (XPath evaluations, FFD requests, Schema validation requests, XSLT requests, single and multiple files, ...)

    Hermann.
  • samanderson
    samanderson
    172 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2012-09-21T06:13:00Z  
    • HermannSW
    • ‏2012-09-19T02:20:27Z
    Forgot to mention that coproc2 is fine for pre v5.0 DataPower firmware.

    Starting with DataPower 5.0.0.x firmware "Interoperability Test Service" allows to do much more than coproc2 can,
    out-of-the box with client on ressource kit, see:
    http://pic.dhe.ibm.com/infocenter/wsdatap/v5r0m0/nav/2_1_0

    (XPath evaluations, FFD requests, Schema validation requests, XSLT requests, single and multiple files, ...)

    Hermann.
    Hi Hermann - Is Interoperability Test Service only runs on unix? is there an IDE for available for (XPath evaluations, FFD requests, Schema validation requests, XSLT requests, single and multiple files, ...)
  • HermannSW
    HermannSW
    4894 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2012-09-21T07:43:53Z  
    Hi Hermann - Is Interoperability Test Service only runs on unix? is there an IDE for available for (XPath evaluations, FFD requests, Schema validation requests, XSLT requests, single and multiple files, ...)
    Hi,

    > Is Interoperability Test Service only runs on unix?
    >
    there is a complete chapter of documentation on "Interoperability Test Service", please have a look:
    http://pic.dhe.ibm.com/infocenter/wsdatap/v5r0m0/nav/2_1_0
    ...
    The Resource Kit provides two test clients, a shell script and a Java program, in the interoperability-service/clients directory:

    The dp-interop-client.sh shell script works on the Linux operating systems or on the Windows operating systems with Cygwin.
    The DPInteropClient.jar Java program can run either on Windows operating systems or on Linux operating systems.
    ...

    > is there an IDE for available for (XPath evaluations, FFD requests, Schema validation requests, XSLT requests, single and multiple files, ...)
    >
    not for that, but for WTX testing which is also available by the Interop Test Service
    (WebSphere Transformation Extender Design Studio).
    This is what you get from ResourceKit for interop testing:
    
    $ cd ResourceKit/IOP-Test $ ll total 8 drwxrwxr-x. 2 stammw stammw 4096 Jul  5 09:44 wtx drwxrwxr-x. 7 stammw stammw 4096 Jul  5 09:44 xslt $ $ ll wtx/  total 8 -r-xr-xr-x. 1 stammw stammw  163 Jul  5 09:44 run.sh -r--r--r--. 1 stammw stammw 3933 Jul  5 09:44 sample.swa $ $ ll xslt
    /* -r--r--r--. 1 stammw stammw 12082 Jul  5 09:44 xslt/DPInteropClient.jar -r--r--r--. 1 stammw stammw 41940 Jul  5 09:44 xslt/DPInteropClient.java -r-xr-xr-x. 1 stammw stammw 28931 Jul  5 09:44 xslt/dp-interop-client.sh xslt/ffd-samples: total 16 drwxrwxr-x. 2 stammw stammw 4096 Jul  5 09:44 dynamic -r-xr-xr-x. 1 stammw stammw  560 Jul  5 09:44 run.sh -r--r--r--. 1 stammw stammw 1865 Jul  5 09:44 sample.html -r--r--r--. 1 stammw stammw 1151 Jul  5 09:44 trxBinary.xsl xslt/schema-samples: total 20 -r--r--r--. 1 stammw stammw  718 Jul  5 09:44 library.xml -r--r--r--. 1 stammw stammw 1754 Jul  5 09:44 library.xsd -r-xr-xr-x. 1 stammw stammw  411 Jul  5 09:44 run.sh -r--r--r--. 1 stammw stammw 2086 Jul  5 09:44 stockQuoteFetcher.wsdl -r--r--r--. 1 stammw stammw  532 Jul  5 09:44 stockQuoteFetcher.xml xslt/xpath-samples: total 8 -r--r--r--. 1 stammw stammw 127 Jul  5 09:44 data.xml -r-xr-xr-x. 1 stammw stammw 440 Jul  5 09:44 run.sh xslt/xslt-samples: total 20 -r--r--r--. 1 stammw stammw   24 Jul  5 09:44 message.xml drwxrwxr-x. 2 stammw stammw 4096 Jul  5 09:44 multi-inputs-1 drwxrwxr-x. 2 stammw stammw 4096 Jul  5 09:44 multi-inputs-2 -r-xr-xr-x. 1 stammw stammw  381 Jul  5 09:44 run.sh -r--r--r--. 1 stammw stammw  284 Jul  5 09:44 toBase64.xsl xslt/zip-samples: total 56 -r--r--r--. 1 stammw stammw 3313 Jul  5 09:44 ffd-dynamic.zip -r--r--r--. 1 stammw stammw 1933 Jul  5 09:44 ffd-samples.zip -r--r--r--. 1 stammw stammw 1094 Jul  5 09:44 ffd-subdir.zip -r--r--r--. 1 stammw stammw    2 Jul  5 09:44 input.dat -r-xr-xr-x. 1 stammw stammw  160 Jul  5 09:44 run-ffd-dynamic.sh -r-xr-xr-x. 1 stammw stammw  144 Jul  5 09:44 run-ffd-samples.sh -r-xr-xr-x. 1 stammw stammw  128 Jul  5 09:44 run-ffd-subdir.sh -r-xr-xr-x. 1 stammw stammw  154 Jul  5 09:44 run-schema-samples-1.sh -r-xr-xr-x. 1 stammw stammw  183 Jul  5 09:44 run-schema-samples-2.sh -r-xr-xr-x. 1 stammw stammw  289 Jul  5 09:44 run-xslt-multi-inputs.sh -r-xr-xr-x. 1 stammw stammw  139 Jul  5 09:44 run-xslt-samples.sh -r--r--r--. 1 stammw stammw 2308 Jul  5 09:44 schema-samples.zip -r--r--r--. 1 stammw stammw 1715 Jul  5 09:44 xslt-multi-inputs.zip -r--r--r--. 1 stammw stammw  596 Jul  5 09:44 xslt-samples.zip $
    


     
    Hermann<myXsltBlog/> <myXsltTweets/>
  • HermannSW
    HermannSW
    4894 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-11T15:00:21Z  
    Hi Craig,

    you were right that coproc2 v1.2 Java client may return nonsense for Non-XML data returned by DataPower.
    Find the really small fix in this posting:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14939311#14939311

     
    Hermann<myXsltBlog/> <myXsltTweets/>
  • chauhan_vin1
    chauhan_vin1
    24 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-12T04:48:40Z  
    • HermannSW
    • ‏2013-02-11T15:00:21Z
    Hi Craig,

    you were right that coproc2 v1.2 Java client may return nonsense for Non-XML data returned by DataPower.
    Find the really small fix in this posting:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14939311#14939311

     
    Hermann<myXsltBlog/> <myXsltTweets/>
    Hi Hermann,

    I am getting this error when I invoke the coproc2 service on the Datapower 4.0.1 using the java code.

    21:39:35 xslt debug 39799852 10.12.28.126 0x80a002ad xmlmgr (default): xslt Compilation Request: attempting to refresh "bad" stylesheet (store:///notfound)
    21:39:35 xslt debug 39799852 10.12.28.126 0x80a002ac xmlmgr (default): xslt Compilation Request: Found in cache (store:///notfound)
    21:39:35 xslt debug 39799852 10.12.28.126 0x80a002aa xmlmgr (default): xslt Compilation Request: Checking cache for URL store:///notfound
    21:39:35 multistep debug 39799852 request 10.12.28.126 0x80c0004e mpgw (coproc2): Stylesheet URL to compile is 'store:///notfound'


    Why is it looking for stylesheet in store:///.?( it has to be in the context xsl)

    I have checked my domain has visibility to the default domain.
  • HermannSW
    HermannSW
    4894 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-12T10:42:05Z  
    Hi Hermann,

    I am getting this error when I invoke the coproc2 service on the Datapower 4.0.1 using the java code.

    21:39:35 xslt debug 39799852 10.12.28.126 0x80a002ad xmlmgr (default): xslt Compilation Request: attempting to refresh "bad" stylesheet (store:///notfound)
    21:39:35 xslt debug 39799852 10.12.28.126 0x80a002ac xmlmgr (default): xslt Compilation Request: Found in cache (store:///notfound)
    21:39:35 xslt debug 39799852 10.12.28.126 0x80a002aa xmlmgr (default): xslt Compilation Request: Checking cache for URL store:///notfound
    21:39:35 multistep debug 39799852 request 10.12.28.126 0x80c0004e mpgw (coproc2): Stylesheet URL to compile is 'store:///notfound'


    Why is it looking for stylesheet in store:///.?( it has to be in the context xsl)

    I have checked my domain has visibility to the default domain.
    Hi Vineet,

    how did you invoke coproc2?

    May you please try the simple sample below and see whether that works or not?
    $ java coproc2 identity.xsl ab.xml http://dp3-l3:2223 ; echo
    <?xml version="1.0" encoding="UTF-8"?>
    <a>1<b>2</b>3</a>
    $ 
    $ cat identity.xsl
    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    >
      <xsl:output method="xml"/>
     
      <xsl:template match="/">
        <xsl:copy-of select="."/>
      </xsl:template>
     
    </xsl:stylesheet>
    $ 
    $ cat ab.xml
    <a>1<b>2</b>3</a>
    $
    


     
    Hermann <myXsltBlog/> <myXsltTweets/>
    Updated on 2014-03-25T02:40:08Z at 2014-03-25T02:40:08Z by iron-man
  • chauhan_vin1
    chauhan_vin1
    24 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-12T17:59:59Z  
    • HermannSW
    • ‏2013-02-12T10:42:05Z
    Hi Vineet,

    how did you invoke coproc2?

    May you please try the simple sample below and see whether that works or not?
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr">$ java coproc2 identity.xsl ab.xml http://dp3-l3:2223 ; echo <?xml version="1.0" encoding="UTF-8"?> <a>1<b>2</b>3</a> $ $ cat identity.xsl <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet> $ $ cat ab.xml <a>1<b>2</b>3</a> $ </pre>

     
    Hermann <myXsltBlog/> <myXsltTweets/>
    I get the same error -

    java coproc2 Identity.xsl ab.xml http://dphost:9031/
    <?xml version="1.0" encoding="UTF-8"?>
    <env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"><env:Body><e
    nv:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from clie
    nt)</faultstring></env:Fault></env:Body></env:Envelope>
    <!-- HTTP response: 500 Internal Server Error -->

    The DP logs states -

    11:00:10 memory-report debug 40345676 10.12.28.126 0x80e00690 mpgw (coproc2): Response Finished: memory used 324580
    11:00:10 mpgw info 40345676 error 10.12.28.126 0x80e000b6 mpgw (coproc2): No match from processing policy 'coproc2_policy' for code '0x00c30002'
    11:00:10 mpgw notice 40345676 10.12.28.126 0x80c0007b stylepolicy (coproc2_policy): No error rule is matched.
    11:00:10 mpgw error 40345676 error 10.12.28.126 0x00c30002 mpgw (coproc2): Compilation error
    11:00:10 multistep error 40345676 request 10.12.28.126 0x80c00009 mpgw (coproc2): request coproc2_policy_rule_0 #2 xform: 'Transforming INPUT with var://context/xsl/ results stored in PIPE' failed: Error opening URL 'store:///notfound'
    11:00:10 xslt error 40345676 10.12.28.126 0x80a002a0 xmlmgr (default): xslt: Request failed to compile. Error opening URL 'store:///notfound' This is referenced by store:///notfound.
    11:00:10 xslt debug 40345676 10.12.28.126 0x80a002a3 xmlmgr (default): xslt Compilation Request: Beginning compilation of URL store:///notfound
    11:00:10 system error 137439235 0x80e0027d Unable to open URL 'store:///notfound'
    11:00:10 xslt debug 40345676 10.12.28.126 0x80a002ad xmlmgr (default): xslt Compilation Request: attempting to refresh "bad" stylesheet (store:///notfound)
    11:00:10 xslt debug 40345676 10.12.28.126 0x80a002ac xmlmgr (default): xslt Compilation Request: Found in cache (store:///notfound)
    11:00:10 xslt debug 40345676 10.12.28.126 0x80a002aa xmlmgr (default): xslt Compilation Request: Checking cache for URL store:///notfound
    11:00:10 multistep debug 40345676 request 10.12.28.126 0x80c0004e mpgw (coproc2): Stylesheet URL to compile is 'store:///notfound'
    11:00:10 multistep info 40345676 request 10.12.28.126 0x80c00002 mpgw (coproc2): rule (coproc2_policy_rule_0): #1 xform: 'Transforming NULL with local:///coproc2.xsl results stored in NULL' completed OK.
    11:00:10 memory-report debug 40345676 request 10.12.28.126 0x80e0068d mpgw (coproc2): Processing finished: memory used 116748
  • chauhan_vin1
    chauhan_vin1
    24 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-12T21:08:17Z  
    I get the same error -

    java coproc2 Identity.xsl ab.xml http://dphost:9031/
    <?xml version="1.0" encoding="UTF-8"?>
    <env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"><env:Body><e
    nv:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from clie
    nt)</faultstring></env:Fault></env:Body></env:Envelope>
    <!-- HTTP response: 500 Internal Server Error -->

    The DP logs states -

    11:00:10 memory-report debug 40345676 10.12.28.126 0x80e00690 mpgw (coproc2): Response Finished: memory used 324580
    11:00:10 mpgw info 40345676 error 10.12.28.126 0x80e000b6 mpgw (coproc2): No match from processing policy 'coproc2_policy' for code '0x00c30002'
    11:00:10 mpgw notice 40345676 10.12.28.126 0x80c0007b stylepolicy (coproc2_policy): No error rule is matched.
    11:00:10 mpgw error 40345676 error 10.12.28.126 0x00c30002 mpgw (coproc2): Compilation error
    11:00:10 multistep error 40345676 request 10.12.28.126 0x80c00009 mpgw (coproc2): request coproc2_policy_rule_0 #2 xform: 'Transforming INPUT with var://context/xsl/ results stored in PIPE' failed: Error opening URL 'store:///notfound'
    11:00:10 xslt error 40345676 10.12.28.126 0x80a002a0 xmlmgr (default): xslt: Request failed to compile. Error opening URL 'store:///notfound' This is referenced by store:///notfound.
    11:00:10 xslt debug 40345676 10.12.28.126 0x80a002a3 xmlmgr (default): xslt Compilation Request: Beginning compilation of URL store:///notfound
    11:00:10 system error 137439235 0x80e0027d Unable to open URL 'store:///notfound'
    11:00:10 xslt debug 40345676 10.12.28.126 0x80a002ad xmlmgr (default): xslt Compilation Request: attempting to refresh "bad" stylesheet (store:///notfound)
    11:00:10 xslt debug 40345676 10.12.28.126 0x80a002ac xmlmgr (default): xslt Compilation Request: Found in cache (store:///notfound)
    11:00:10 xslt debug 40345676 10.12.28.126 0x80a002aa xmlmgr (default): xslt Compilation Request: Checking cache for URL store:///notfound
    11:00:10 multistep debug 40345676 request 10.12.28.126 0x80c0004e mpgw (coproc2): Stylesheet URL to compile is 'store:///notfound'
    11:00:10 multistep info 40345676 request 10.12.28.126 0x80c00002 mpgw (coproc2): rule (coproc2_policy_rule_0): #1 xform: 'Transforming NULL with local:///coproc2.xsl results stored in NULL' completed OK.
    11:00:10 memory-report debug 40345676 request 10.12.28.126 0x80e0068d mpgw (coproc2): Processing finished: memory used 116748
    Hermann,

    I believe the coproc2 mpg has to be created in the default domain.? - I created it in a separate domain may be why its not working.

    Thanks,
    Vineet
  • chauhan_vin1
    chauhan_vin1
    24 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-12T21:08:59Z  
    • HermannSW
    • ‏2013-02-12T10:42:05Z
    Hi Vineet,

    how did you invoke coproc2?

    May you please try the simple sample below and see whether that works or not?
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr">$ java coproc2 identity.xsl ab.xml http://dp3-l3:2223 ; echo <?xml version="1.0" encoding="UTF-8"?> <a>1<b>2</b>3</a> $ $ cat identity.xsl <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet> $ $ cat ab.xml <a>1<b>2</b>3</a> $ </pre>

     
    Hermann <myXsltBlog/> <myXsltTweets/>
    Hermann,

    I believe the coproc2 mpg has to be created in the default domain.? - I created it in a separate domain may be why its not working.

    Thanks,
    Vineet
  • chauhan_vin1
    chauhan_vin1
    24 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-13T00:03:23Z  
    • HermannSW
    • ‏2013-02-12T10:42:05Z
    Hi Vineet,

    how did you invoke coproc2?

    May you please try the simple sample below and see whether that works or not?
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr">$ java coproc2 identity.xsl ab.xml http://dp3-l3:2223 ; echo <?xml version="1.0" encoding="UTF-8"?> <a>1<b>2</b>3</a> $ $ cat identity.xsl <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet> $ $ cat ab.xml <a>1<b>2</b>3</a> $ </pre>

     
    Hermann <myXsltBlog/> <myXsltTweets/>
    I changed the context variable to var://context/coproc2/xsl - But now it gives another error "Unable to Open URL"


    16:58:57 memory-report debug 92415382 xx.xx.xx.xxx 0x80e00690 mpgw (coproc2): Response Finished: memory used 591236
    16:58:57 mpgw info 92415382 error xx.xx.xx.xxx 0x80e000b6 mpgw (coproc2): No match from processing policy 'coproc2_policy' for code '0x00c30002'
    16:58:57 mpgw notice 92415382 xx.xx.xx.xxx 0x80c0007b stylepolicy (coproc2_policy): No error rule is matched.
    16:58:57 mpgw error 92415382 error xx.xx.xx.xxx 0x00c30002 mpgw (coproc2): Compilation error
    16:58:57 multistep error 92415382 request xx.xx.xx.xxx 0x80c00009 mpgw (coproc2): request coproc2_policy_rule_0 #2 xform: 'Transforming INPUT with var://context/coproc2/xsl results stored in PIPE' failed: Unable to open URL
    16:58:57 xslt error 92415382 xx.xx.xx.xxx 0x80a002a0 xmlmgr (default): xslt: Request failed to compile. Unable to open URL This is referenced by <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>.
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002a3 xmlmgr (default): xslt Compilation Request: Beginning compilation of URL <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002ad xmlmgr (default): xslt Compilation Request: attempting to refresh "bad" stylesheet (<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>)
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002ac xmlmgr (default): xslt Compilation Request: Found in cache (<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>)
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002aa xmlmgr (default): xslt Compilation Request: Checking cache for URL <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>
    16:58:57 multistep debug 92415382 request xx.xx.xx.xxx 0x80c0004e mpgw (coproc2): Stylesheet URL to compile is '<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>'
    16:58:57 multistep info 92415382 request xx.xx.xx.xxx 0x80c00002 mpgw (coproc2): rule (coproc2_policy_rule_0): #1 xform: 'Transforming NULL with local:///coproc2.xsl results stored in NULL' completed OK.
    16:58:57 memory-report debug 92415382 request xx.xx.xx.xxx 0x80e0068d mpgw (coproc2): Processing finished: memory used 359284
    16:58:57 mpgw info 92415382 xx.xx.xx.xxx 0x80e00115 mpgw (coproc2): Will not process backside due to var://service/mpgw/skip-backside
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002ac xmlmgr (default): xslt Compilation Request: Found in cache (local:///coproc2.xsl)
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002aa xmlmgr (default): xslt Compilation Request: Checking cache for URL local:///coproc2.xsl
    16:58:57 multistep debug 92415382 request xx.xx.xx.xxx 0x80c0004e mpgw (coproc2): Stylesheet URL to compile is 'local:///coproc2.xsl'
  • HermannSW
    HermannSW
    4894 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-13T00:19:50Z  
    I changed the context variable to var://context/coproc2/xsl - But now it gives another error "Unable to Open URL"


    16:58:57 memory-report debug 92415382 xx.xx.xx.xxx 0x80e00690 mpgw (coproc2): Response Finished: memory used 591236
    16:58:57 mpgw info 92415382 error xx.xx.xx.xxx 0x80e000b6 mpgw (coproc2): No match from processing policy 'coproc2_policy' for code '0x00c30002'
    16:58:57 mpgw notice 92415382 xx.xx.xx.xxx 0x80c0007b stylepolicy (coproc2_policy): No error rule is matched.
    16:58:57 mpgw error 92415382 error xx.xx.xx.xxx 0x00c30002 mpgw (coproc2): Compilation error
    16:58:57 multistep error 92415382 request xx.xx.xx.xxx 0x80c00009 mpgw (coproc2): request coproc2_policy_rule_0 #2 xform: 'Transforming INPUT with var://context/coproc2/xsl results stored in PIPE' failed: Unable to open URL
    16:58:57 xslt error 92415382 xx.xx.xx.xxx 0x80a002a0 xmlmgr (default): xslt: Request failed to compile. Unable to open URL This is referenced by <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>.
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002a3 xmlmgr (default): xslt Compilation Request: Beginning compilation of URL <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002ad xmlmgr (default): xslt Compilation Request: attempting to refresh "bad" stylesheet (<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>)
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002ac xmlmgr (default): xslt Compilation Request: Found in cache (<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>)
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002aa xmlmgr (default): xslt Compilation Request: Checking cache for URL <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>
    16:58:57 multistep debug 92415382 request xx.xx.xx.xxx 0x80c0004e mpgw (coproc2): Stylesheet URL to compile is '<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>'
    16:58:57 multistep info 92415382 request xx.xx.xx.xxx 0x80c00002 mpgw (coproc2): rule (coproc2_policy_rule_0): #1 xform: 'Transforming NULL with local:///coproc2.xsl results stored in NULL' completed OK.
    16:58:57 memory-report debug 92415382 request xx.xx.xx.xxx 0x80e0068d mpgw (coproc2): Processing finished: memory used 359284
    16:58:57 mpgw info 92415382 xx.xx.xx.xxx 0x80e00115 mpgw (coproc2): Will not process backside due to var://service/mpgw/skip-backside
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002ac xmlmgr (default): xslt Compilation Request: Found in cache (local:///coproc2.xsl)
    16:58:57 xslt debug 92415382 xx.xx.xx.xxx 0x80a002aa xmlmgr (default): xslt Compilation Request: Checking cache for URL local:///coproc2.xsl
    16:58:57 multistep debug 92415382 request xx.xx.xx.xxx 0x80c0004e mpgw (coproc2): Stylesheet URL to compile is 'local:///coproc2.xsl'
    Hi,

    you did change the coproc2 service (at least the port number which is 2223 normally).

    May you please import the original coproc2 service unchanged ?
    That should work.
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14590913#14590913

     
    Hermann<myXsltBlog/> <myXsltTweets/>
  • chauhan_vin1
    chauhan_vin1
    24 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-13T17:59:07Z  
    • HermannSW
    • ‏2013-02-13T00:19:50Z
    Hi,

    you did change the coproc2 service (at least the port number which is 2223 normally).

    May you please import the original coproc2 service unchanged ?
    That should work.
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14590913#14590913

     
    Hermann<myXsltBlog/> <myXsltTweets/>
    Hermann,

    I dont have the permissions to import anything into the default domain. Can you walk me the steps on how to setup the MPG manually.

    One thing I observed. When I store the XSL in the context - var://context/xsl , the subsequent transform is not able to read it as its coming as null.
    When I store it in var://context/contextName/xsl - its working but failing to compile the syle sheet from the URL :

    16:58:57 xslt error 92415382 xx.xx.xx.xxx 0x80a002a0 xmlmgr (default): xslt: Request failed to compile. Unable to open URL This is referenced by <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>.
  • HermannSW
    HermannSW
    4894 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-14T08:39:08Z  
    Hermann,

    I dont have the permissions to import anything into the default domain. Can you walk me the steps on how to setup the MPG manually.

    One thing I observed. When I store the XSL in the context - var://context/xsl , the subsequent transform is not able to read it as its coming as null.
    When I store it in var://context/contextName/xsl - its working but failing to compile the syle sheet from the URL :

    16:58:57 xslt error 92415382 xx.xx.xx.xxx 0x80a002a0 xmlmgr (default): xslt: Request failed to compile. Unable to open URL This is referenced by <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:output method="xml"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>.
    Hi Vineet,

    >
    > I dont have the permissions to import anything into the default domain.
    >
    you do not need to -- you can import coproc2.zip into any application domain, its an export and not a backup.

    >Can you walk me the steps on how to setup the MPG manually.
    >
    The detailed steps are here:
    https://www.ibm.com/developerworks/forums/thread.jspa?threadID=363570#14590915
    > One thing I observed. When I store the XSL in the context - var://context/xsl , the subsequent transform is not able to read it as its coming as null.
    > When I store it in var://context/contextName/xsl - its working but failing to compile the syle sheet from the URL :
    >
    It is difficult to setup, either import into application domain, or exactly follow the steps referred to above.
    It would be really helpful if you do not change anything until it works correctly first -- then you can change whatever you want.

     
    Hermann<myXsltBlog/> <myXsltTweets/>
  • Jaango
    Jaango
    267 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-18T15:26:27Z  
    • HermannSW
    • ‏2013-02-14T08:39:08Z
    Hi Vineet,

    >
    > I dont have the permissions to import anything into the default domain.
    >
    you do not need to -- you can import coproc2.zip into any application domain, its an export and not a backup.

    >Can you walk me the steps on how to setup the MPG manually.
    >
    The detailed steps are here:
    https://www.ibm.com/developerworks/forums/thread.jspa?threadID=363570#14590915
    > One thing I observed. When I store the XSL in the context - var://context/xsl , the subsequent transform is not able to read it as its coming as null.
    > When I store it in var://context/contextName/xsl - its working but failing to compile the syle sheet from the URL :
    >
    It is difficult to setup, either import into application domain, or exactly follow the steps referred to above.
    It would be really helpful if you do not change anything until it works correctly first -- then you can change whatever you want.

     
    Hermann<myXsltBlog/> <myXsltTweets/>
    With coproc version 3, debugging and troubleshooting has become very flexible :). Can we use coproc service to get the trace output(as a html file), for an xsl with xml input, may be by using compile options policy of the coproc service?

    Just want to know if we can use coproc for troubleshooting at xslt code level as well.
  • HermannSW
    HermannSW
    4894 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-19T16:48:53Z  
    • Jaango
    • ‏2013-02-18T15:26:27Z
    With coproc version 3, debugging and troubleshooting has become very flexible :). Can we use coproc service to get the trace output(as a html file), for an xsl with xml input, may be by using compile options policy of the coproc service?

    Just want to know if we can use coproc for troubleshooting at xslt code level as well.
    Hi Maneesh,

    > With coproc version 3, debugging and troubleshooting has become very flexible :).
    > Can we use coproc service to get the trace output(as a html file), for an xsl with xml input,
    >
    yes.

    > may be by using compile options policy of the coproc service?
    >
    First I was sceptical, but YES, just changing the compile options policy does give you the trace output!
    (of course yo can have several coproc2 services, one "normal", one for trace output, one for BIG XML data, ...)

    Why was I sceptical?
    Because coproc2 service rule consists of two Transform actions, and the second is the one you are interested in.
    But the first Transform action is from NULL to NULL, so you get the output you are interested in!

    Just add a Debug Rule to the Compile Options of coproc2's XML manager -- that's it.

    
    $ coproc2 double.xsl ab.xml http:
    //firestar:2223 -s > double.xsl.html $ $ cat ab.xml <a>1<b>2</b>3</a> $
    


    And this is "double.xsl.html" in a browser, and double.xsl code listing with line numbers for refrence:

    > Just want to know if we can use coproc for troubleshooting at xslt code level as well.
    >
    Yes -- thanks for that interesting idea!

     
    Hermann<myXsltBlog/> <myXsltTweets/>
  • Jaango
    Jaango
    267 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-20T09:59:43Z  
    • HermannSW
    • ‏2013-02-19T16:48:53Z
    Hi Maneesh,

    > With coproc version 3, debugging and troubleshooting has become very flexible :).
    > Can we use coproc service to get the trace output(as a html file), for an xsl with xml input,
    >
    yes.

    > may be by using compile options policy of the coproc service?
    >
    First I was sceptical, but YES, just changing the compile options policy does give you the trace output!
    (of course yo can have several coproc2 services, one "normal", one for trace output, one for BIG XML data, ...)

    Why was I sceptical?
    Because coproc2 service rule consists of two Transform actions, and the second is the one you are interested in.
    But the first Transform action is from NULL to NULL, so you get the output you are interested in!

    Just add a Debug Rule to the Compile Options of coproc2's XML manager -- that's it.

    <pre class="jive-pre"> $ coproc2 double.xsl ab.xml http: //firestar:2223 -s > double.xsl.html $ $ cat ab.xml <a>1<b>2</b>3</a> $ </pre>

    And this is "double.xsl.html" in a browser, and double.xsl code listing with line numbers for refrence:

    > Just want to know if we can use coproc for troubleshooting at xslt code level as well.
    >
    Yes -- thanks for that interesting idea!

     
    Hermann<myXsltBlog/> <myXsltTweets/>
    Hermann,

    Thanks. Due to some reasons, I am getting the below error while defining a debug rule(with * for matching all). Also the datapower box kicks me out from the webgui, after running the command, have to relog in again.
    C:\Program Files\Java\jre1.6.0_20\bin>java coproc2 double.xsl ab.xml http://mybox:2223 >double.xsl.html
    Exception in thread "main" java.net.ConnectException: Connection refused: connec
    t
            at java.net.PlainSocketImpl.socketConnect(Native Method)
            at java.net.PlainSocketImpl.doConnect(Unknown Source)
            at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
            at java.net.PlainSocketImpl.connect(Unknown Source)
            at java.net.SocksSocketImpl.connect(Unknown Source)
            at java.net.Socket.connect(Unknown Source)
            at java.net.Socket.connect(Unknown Source)
            at sun.net.NetworkClient.doConnect(Unknown Source)
            at sun.net.www.http.HttpClient.openServer(Unknown Source)
            at sun.net.www.http.HttpClient.openServer(Unknown Source)
            at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
            at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
            at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
    urce)
            at java.net.HttpURLConnection.getResponseCode(Unknown Source)
            at coproc2.main(coproc2.java:154)
    

    If I am not using a debug rule, it just runs fine.
    Updated on 2014-03-25T02:39:42Z at 2014-03-25T02:39:42Z by iron-man
  • HermannSW
    HermannSW
    4894 Posts

    Re: coproc2 v1.3 -- version to upload multiple files

    ‏2013-02-20T12:07:20Z  
    • Jaango
    • ‏2013-02-20T09:59:43Z
    Hermann,

    Thanks. Due to some reasons, I am getting the below error while defining a debug rule(with * for matching all). Also the datapower box kicks me out from the webgui, after running the command, have to relog in again.
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr">C:\Program Files\Java\jre1.6.0_20\bin>java coproc2 double.xsl ab.xml http://mybox:2223 >double.xsl.html Exception in thread "main" java.net.ConnectException: Connection refused: connec t at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(Unknown Source) at java.net.PlainSocketImpl.connectToAddress(Unknown Source) at java.net.PlainSocketImpl.connect(Unknown Source) at java.net.SocksSocketImpl.connect(Unknown Source) at java.net.Socket.connect(Unknown Source) at java.net.Socket.connect(Unknown Source) at sun.net.NetworkClient.doConnect(Unknown Source) at sun.net.www.http.HttpClient.openServer(Unknown Source) at sun.net.www.http.HttpClient.openServer(Unknown Source) at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source) at sun.net.www.http.HttpClient.parseHTTP(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So urce) at java.net.HttpURLConnection.getResponseCode(Unknown Source) at coproc2.main(coproc2.java:154) </pre>
    If I am not using a debug rule, it just runs fine.
    Hi,

    > Thanks. Due to some reasons, I am getting the below error while defining a debug rule(with * for matching all).
    > Also the datapower box kicks me out from the webgui, after running the command, have to relog in again.
    > ...
    >...
    > If I am not using a debug rule, it just runs fine.

    It seems that your firmware level is below "3.8.0.18, 3.8.1.18, 3.8.2.9, 4.0.1.6 and 4.0.2.2".
    I fixed a firmware restart with Debug Rule in APAR IC78912 in December 2011 fixpack.
    You need to upgrade the firmware in that case, there is no workaround.

    More information on stylesheet tracing can be found here:
    https://www.ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/entry/stylesheet_profiling1

     
    Hermann<myXsltBlog/> <myXsltTweets/>