Topic
9 replies Latest Post - ‏2013-07-23T15:30:05Z by kpc_pdx
SystemAdmin
SystemAdmin
6772 Posts
ACCEPTED ANSWER

Pinned topic Storing file at run time in local disk

‏2011-07-07T02:28:49Z |
Hi,

I'm having a scheduled rule which triggers each day and do a web service call to an another interface. Now, the response of that interface has to be stored in local disk of the box as it contains some master data which will be used by other service components.
Can I use XMI/SOMA internally? :) Or any other idea? Pl help....
Updated on 2011-07-07T13:25:54Z at 2011-07-07T13:25:54Z by Liv2luv
  • nepomuk
    nepomuk
    89 Posts
    ACCEPTED ANSWER

    Re: Storing file at run time in local disk

    ‏2011-07-07T08:14:03Z  in response to SystemAdmin
    I have a similar problem to solve. I don't think you can use SOMA internally, as far as I know the file system can't be updated from within datapower. I will be using message broker to send the SOMA messages to datapower but you should be able to use any programming environment to process the web service response, build the SOMA message and send it back in to datapower.
  • HermannSW
    HermannSW
    4155 Posts
    ACCEPTED ANSWER

    Re: Storing file at run time in local disk

    ‏2011-07-07T08:56:59Z  in response to SystemAdmin
    Hi,
    >
    > I'm having a scheduled rule which triggers each day and do a web service call to an another interface. Now, the response of that
    > interface has to be stored in local disk of the box as it contains some master data which will be used by other service components.
    > Can I use XMI/SOMA internally? :) Or any other idea? Pl help....

    yes, you can, see posting "Accessing XML Management interface from within a stylesheet"
    https://www.ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/entry/accessing_xml_management_interface_from_within_a_stylesheet38
    http://tinyurl.com/6yk787p (because URL is too long for dw Forum SW)

    You can use set-file operation for that. The file needs to be presented base64-encoded.
    See stylesheet "set-file.xsl" below on how to create the XML management request:
    
    $ wget -q -O - http:
    //dp4-l3/ab.xml <a>1<b>2</b>3</a> $ $ java coproc2 set-file.xsl ab.xml http:
    //cosmopolitan:2223 | tidy -q -xml <?xml version=
    "1.0" encoding=
    "utf-8"?> <soapenv:Envelope xmlns:dp=
    "http://www.datapower.com/schemas/management" xmlns:soapenv=
    "http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <dp:request domain=
    "default"> <dp:set-file name=
    "local:///myTestfile"> PGE+MTxiPjI8L2I+MzwvYT4=</dp:set-file> </dp:request> </soapenv:Body> </soapenv:Envelope>   $ cat set-file.xsl <xsl:stylesheet version=
    "1.0" xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:dpe=
    "http://www.datapower.com/extensions" xmlns:dp=
    "http://www.datapower.com/schemas/management" xmlns:soapenv=
    "http://schemas.xmlsoap.org/soap/envelope/" extension-element-prefixes=
    "dpe" > <xsl:template match=
    "/"> <xsl:variable name=
    "file"> <dpe:url-open target=
    "http://dp4-l3.boeblingen.de.ibm.com/ab.xml"/> </xsl:variable>   <xsl:variable name=
    "serialized"> <dpe:serialize select=
    "$file" omit-xml-decl=
    "yes"/> </xsl:variable>   <xsl:variable name=
    "cmd"> <soapenv:Envelope> <soapenv:Body> <dp:request domain=
    "default"> <dp:set-file name=
    "local:///myTestfile"> <xsl:value-of select=
    "dpe:encode($serialized,'base-64')"/> </dp:set-file> </dp:request> </soapenv:Body> </soapenv:Envelope> </xsl:variable>   <xsl:copy-of select=
    "$cmd"/> </xsl:template> </xsl:stylesheet> $
    


    It does not match your requirement, but just in case you need to get and store Non-XML files, this set-file.xsl modification does it:
    
    ... <xsl:variable name=
    "file"> <dpe:url-open response=
    "binaryNode" target=
    "http://dp4-l3.boeblingen.de.ibm.com/binaryFile"/> </xsl:variable> <!-- <xsl:variable name=
    "serialized"> <dpe:serialize select=
    "$file" omit-xml-decl=
    "yes"/> </xsl:variable> --> <xsl:variable name=
    "cmd"> <soapenv:Envelope> <soapenv:Body> <dp:request domain=
    "default"> <dp:set-file name=
    "local:///myTestfile"> <xsl:value-of select=
    "dpe:binary-encode($file)"/> </dp:set-file> </dp:request> </soapenv:Body> </soapenv:Envelope> </xsl:variable> ...
    


    Btw, shell-script "set-file" below is handy for creating the request for sending eg. by doSoma
    (see https://www.ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/tags/dosoma):
    
    $ cat testfile test $ $ set-file local:
    ///myTestfile testfile default <?xml version=
    "1.0" encoding=
    "UTF-8"?> <soapenv:Envelope xmlns:soapenv=
    "http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <dp:request domain=
    "default" xmlns:dp=
    "http://www.datapower.com/schemas/management"> <dp:set-file name=
    "local:///myTestfile"> dGVzdAo= </dp:set-file> </dp:request> </soapenv:Body> </soapenv:Envelope> $ $ cat set-file #!/bin/bash echo 
    '<?xml version="1.0" encoding="UTF-8"?>' echo 
    '<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">' echo 
    '<soapenv:Body>' echo 
    '<dp:request domain="'$3
    '" xmlns:dp="http://www.datapower.com/schemas/management">' echo 
    '<dp:set-file name="'$1
    '">' base64 $2 echo 
    '</dp:set-file>' echo 
    '</dp:request>' echo 
    '</soapenv:Body>' echo 
    '</soapenv:Envelope>' $
    


     
    Hermann<myXsltBlog/>
    • SystemAdmin
      SystemAdmin
      6772 Posts
      ACCEPTED ANSWER

      Re: Storing file at run time in local disk

      ‏2011-07-07T09:47:21Z  in response to HermannSW
      Thanks Hermann for your input.

      I had seen your paper on this before my post and my approach was to use SOMA internally, but was curious to know about any other xslt/dp functions available to perform this. :)
      • Liv2luv
        Liv2luv
        573 Posts
        ACCEPTED ANSWER

        Re: Storing file at run time in local disk

        ‏2011-07-07T13:25:54Z  in response to SystemAdmin
        >..my approach was to use SOMA internally, but was curious to know about any other xslt/dp functions available to perform this

        I would try to make some thing similar as shown in the Blog post of Hermann. If you encounter any issues in specific that may be easy to address, I guess.

        Thanks.
    • HermannSW
      HermannSW
      4155 Posts
      ACCEPTED ANSWER

      Re: Storing file at run time in local disk

      ‏2013-04-26T07:41:45Z  in response to HermannSW

      <EDIT>
      It is so painful to repair {code}...{code} section defacement after forum software migration ...

      And trying to edit old entries results in error message -- just trying to append as "new" posting now.

      So this new entry can be edited.

      What I learned sofar:
      Mark code sections "pre" and "code" (below edit area) and as "blockquote" (toolbar).

      Still no idea how to prevent "crazy" starting of new lines in the middle of code sections ...

      And what is absolutely horrible:
      After "save" EVERYTHING in code sections seems to be broken, both in Firefox as well as Chrome browsers.

      But don't panic, just "reload" the page in browser and the display is fine again ... :-(

      </EDIT>

      Hi,
      >
      > I'm having a scheduled rule which triggers each day and do a web service call to an another interface. Now, the response of that
      > interface has to be stored in local disk of the box as it contains some master data which will be used by other service components.
      > Can I use XMI/SOMA internally? :) Or any other idea? Pl help....

      yes, you can, see posting "Accessing XML Management interface from within a stylesheet"
      https://www.ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/entry/accessing_xml_management_interface_from_within_a_stylesheet38
      http://tinyurl.com/6yk787p (because URL is too long for dw Forum SW)

      You can use set-file operation for that. The file needs to be presented base64-encoded.
      See stylesheet "set-file.xsl" below on how to create the XML management request:

      $ wget -q -O - 
      http://dp4-l3/ab.xml
      <a>1<b>2</b>3</a>
      $
      $ java coproc2 set-file.xsl ab.xml 
      http://cosmopolitan:2223 | tidy -q -xml
      <?xml version="1.0" encoding="utf-8"?>
      <soapenv:Envelope
      xmlns:dp="http://www.datapower.com/schemas/management"
      xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
        <soapenv:Body>
          <dp:request domain="default">
            <dp:set-file name="local:///myTestfile">
            PGE+MTxiPjI8L2I+MzwvYT4=</dp:set-file>
          </dp:request>
        </soapenv:Body>
      </soapenv:Envelope>
      
      $ cat set-file.xsl
      <xsl:stylesheet version="1.0"
        xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
        xmlns:dpe="http://www.datapower.com/extensions"
        xmlns:dp="http://www.datapower.com/schemas/management"
        xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
        extension-element-prefixes="dpe"
      >
        <xsl:template match="/">
          <xsl:variable name="file">
            <dpe:url-open target="http://dp4-l3.boeblingen.de.ibm.com/ab.xml"/>
          </xsl:variable>
      
          <xsl:variable name="serialized">
            <dpe:serialize select="$file" omit-xml-decl="yes"/>
          </xsl:variable>
      
          <xsl:variable name="cmd">
            <soapenv:Envelope>
              <soapenv:Body>
                <dp:request domain="default">
                  <dp:set-file name="local:///myTestfile">
                    <xsl:value-of select="dpe:encode($serialized,'base-64')"/>
                  </dp:set-file>
                </dp:request>
              </soapenv:Body>
            </soapenv:Envelope>
          </xsl:variable>
      
          <xsl:copy-of select="$cmd"/>
        </xsl:template>
      </xsl:stylesheet>
      
      $
      


      It does not match your requirement, but just in case you need to get and store Non-XML files, this set-file.xsl modification does it:

          ...
               <xsl:variable name="file">
                 <dpe:url-open response="binaryNode"
                   target="http://dp4-l3.boeblingen.de.ibm.com/binaryFile"
                 />     
               </xsl:variable>     
               <!--       
                 <xsl:variable name="serialized">
                   <dpe:serialize select="$file" omit-xml-decl="yes"/>
                 </xsl:variable>
               -->
               <xsl:variable name="cmd">
                 <soapenv:Envelope>
                   <soapenv:Body>
                     <dp:request domain="default">
                       <dp:set-file name="local:///myTestfile">
                         <xsl:value-of select="dpe:binary-encode($file)"/>
                       </dp:set-file>
                     </dp:request>
                   </soapenv:Body>
                 </soapenv:Envelope>
               </xsl:variable>
           ...


      Btw, shell-script "set-file" below is handy for creating the request for sending eg. by doSoma
      (see https://www.ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/tags/dosoma):

      
      $ cat testfile 
      test 
      $ 
      $ set-file local:
      ///myTestfile testfile default 
      <?xml version="1.0" encoding="UTF-8"?>
      <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
      <soapenv:Body>
      <dp:request domain="default" xmlns:dp="http://www.datapower.com/schemas/management">
      <dp:set-file name="local:///myTestfile">
      dGVzdAo=
      </dp:set-file>
      </dp:request>
      </soapenv:Body>
      </soapenv:Envelope>
      
      $ 
      $ cat set-file 
      #!/bin/bash
      echo '<?xml version="1.0" encoding="UTF-8"?>'
      echo '<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">'
      echo '<soapenv:Body>'
      echo '<dp:request domain="'$3'" xmlns:dp="http://www.datapower.com/schemas/management">'
      echo '<dp:set-file name="'$1'">'
      base64 $2
      echo '</dp:set-file>'
      echo '</dp:request>'
      echo '</soapenv:Body>'
      echo '</soapenv:Envelope>'
      
      $
      

       

      Hermann<myXsltBlog/>

       

      Updated on 2013-04-26T22:08:13Z at 2013-04-26T22:08:13Z by HermannSW
  • HermannSW
    HermannSW
    4155 Posts
    ACCEPTED ANSWER

    Re: Storing file at run time in local disk

    ‏2011-07-07T09:55:34Z  in response to SystemAdmin

    I forgot to mention how to make use of set-file shell-script with doSoma tool.

    Here file "testfile" gets stored as "local:///myTestfile2" in "default" domain ...

    $ set-file local://myTestfile2 testfile default | \
    > doSoma admin - dp3-l3:5550 | \
    > tidy -q -xml
    Enter host password for user 'admin':
    <?xml version="1.0" encoding="utf-8"?>
    <env:Envelope
    xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
      <env:Body>
        <dp:response
        xmlns:dp="http://www.datapower.com/schemas/management">
          <dp:timestamp>2013-07-12T04:47:26-04:00</dp:timestamp>
          <dp:result>OK</dp:result>
        </dp:response>
      </env:Body>
    </env:Envelope>

    $


    Hermann<myXsltBlog/>

     

    Updated on 2013-07-12T09:03:28Z at 2013-07-12T09:03:28Z by HermannSW
    • kpc_pdx
      kpc_pdx
      22 Posts
      ACCEPTED ANSWER

      Re: Storing file at run time in local disk

      ‏2013-07-11T14:22:12Z  in response to HermannSW


      Hi Hermann,

      I've found myself back to this article, from our previous discussion on FTP and writing files locally and to FTP.

      I'm using the set-file.xsl above to capture a Datapower context variable from an Extract from XPath action. This xsl thus sits in an error rule.

      When we get an error, the error rule will/should send the Datapower context variable (which contains most of the SOAP request nodes) to an FTP server as multiple files each instance. So let's say the say filename with dates/times making each file unique.

      Question is:  Is this url-open I see making an XML mgmt call or is it retrieving  the 'ab.xml' file?

      My plan was to capture the context variable and write to an xml file on the FTP server.

      Thanks,

      Kirby

      • HermannSW
        HermannSW
        4155 Posts
        ACCEPTED ANSWER

        Re: Storing file at run time in local disk

        ‏2013-07-12T09:11:01Z  in response to kpc_pdx

        Hi Kirby,

        > Question is:  Is this url-open I see making an XML mgmt call or is it retrieving  the 'ab.xml' file?
        >
        it is just retrieving  the 'ab.xml' file from remote (dp4-l3).

        Try first to log your context in the error rule so that you are sure you have what you want.

        Then use <dp:url-open> with FTP URLs to store on FTP server:
        http://pic.dhe.ibm.com/infocenter/wsdatap/v5r0m0/index.jsp?topic=%2Fcom.ibm.dp.xi.doc%2Fextensionfunctions36.htm

         

        Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

        • This reply was deleted by kpc_pdx 2013-07-22T20:01:32Z. Reason for deletion: revising the reposting code relevant to local disk discussion
        • kpc_pdx
          kpc_pdx
          22 Posts
          ACCEPTED ANSWER

          Re: Storing file at run time in local disk

          ‏2013-07-23T15:30:05Z  in response to HermannSW


          Thanks Hermann,

          I ended up with the following below. It builds the files locally without any mgmt calls, makes them unique, then sends
          them ftp. There is a log action after the Transform Action which logs the variable of REQUEST.

              <xsl:template match="/">

                  <xsl:variable name="time-value" select="dpe:time-value()"/>
                  <xsl:variable name="fileDate" select="concat('ErrorRequest_', $time-value)"/>
                  <xsl:variable name="fileName" select="concat($fileDate,'.xml')"/>
                  <dpe:set-variable name="'var://context/EXTRACT/REQUEST'" value="concat('ftp://datapower:datapower@drfsfilp2.corp.jmfamily.com/',$fileName)"/>

              </xsl:template>

           

          Updated on 2013-07-23T15:30:33Z at 2013-07-23T15:30:33Z by kpc_pdx