IBM Support

Using the Compression Service without using the File System.

Technical Blog Post


Abstract

Using the Compression Service without using the File System.

Body

Sterling B2B Integrator has a Compression Service which is used to zip files. Let's say we want to collect files from a directory on the File System, compress the files into to a zip file, and write the zipped file back to the File System.  The Compression Service lends itself to working with the File System, and we can do this easily:

<process name="compressFiles">
  <sequence>
    <operation name="FileSystem">
      <participant name="FileSystem"/>
      <output message="outputMessage">
        <assign to="Action">FS_COLLECT</assign>
        <assign to="collectionFolder">/opt/si/install/nonedioutbound/files2compress</assign>
        <assign to="collectMultiple">true</assign>
        <assign to="." from="*"></assign>
      </output>
      <input message="inputMessage">
        <assign to="." from="*"></assign>
      </input>
    </operation>    
    <operation name="Compress">
      <participant name="CompressionService"/>
      <output message="outputMessage">
        <assign to="." from="*"></assign>
        <assign to="compression_action">compress</assign>
        <assign to="compressed_filename">DeflatedFile.zip</assign>
        <assign to="compression_level">9</assign>
        <assign to="compression_type">Deflate</assign>
      </output>
      <input message="inputMessage">
        <assign to="." from="*"></assign>
      </input>
    </operation>
    <operation name="FileSystem">
      <participant name="FileSystem"/>
      <output message="outputMessage">
        <assign to="Action">FS_EXTRACT</assign>
        <assign to="extractionFolder">/opt/si/install/nonedioutbound/compressedFiles</assign>
        <assign to="." from="*"></assign>
      </output>
      <input message="inputMessage">
        <assign to="." from="*"></assign>
      </input>
    </operation>  
  </sequence>
</process>

The File System Adapter works hand-in-glove with the Compression Service to handle the operation elegantly. The File System Adapter creates uniquely named nodes in the ProcessData with attributes for the file names:

<?xml version="1.0" encoding="UTF-8"?>
<ProcessData>
  <FSA_Document1 SCIObjectID="816738156763577d8node1" filename="acme44.edi"/>
  <FSA_Document2 SCIObjectID="346739156763577d8node1" filename="acme45.edi"/>
  <FSA_Document3 SCIObjectID="526740156763577d8node1" filename="acme43.edi"/>
  <FSA_DocumentCount>3</FSA_DocumentCount>
  <PrimaryDocument SCIObjectID="426754156763577d8node1"/>
</ProcessData>

However, what if our file source is not a directory on the File System, but a Sterling B2B Integrator Mailbox?  Can we still accomplish this goal?  The short answer is yes.  After all, we can extract the files from the mailbox, write the files to the File System, collect the files from the File System and compress the lot.  This will certainly work, but there are obvious disadvantages to this approach.  For one thing, writing files to and from the File System is expensive, requiring lots of I/O.  Eliminating the File System and building the list of messages in the ProcessData would be more efficient.  We might want to try an approach like this:
 

  1. Query the mailbox for extractable messages.
  2. Extract each message.
  3. Write the message to a node in the ProcessData.
  4. Repeat the process until all messages have been extracted.
  5. Compress the nodes into a single zip file.
  6. Delete the extracted messages from the mailbox.


In BPML, the result might be:
 

<process name="compressFiles">
  <rule name="atLeastOneMessage">
    <condition>count(/ProcessData/Message) &gt; 0</condition>
  </rule>
  <rule name="atLeastOneSavedNode">
    <condition>count(/ProcessData/*[starts-with(name(), /ProcessData/savedPrefix/text())]) &gt; 0</condition>
  </rule>
  <sequence name="begin">
    <assign to="inMailbox">/compress/inbox</assign>
    <assign to="outMailbox">/compress/outbox</assign>
    <!--
     as each message becomes the PrimaryDocument, save it to a
     node with a naming convention of savedPrefix + savedCount
    -->
    <assign to="savedPrefix">Document</assign>
    <assign to="savedCount">0</assign>
    <operation name="Mailbox Query Service">
      <participant name="MailboxQuery"/>
      <output message="QueryRequest">
        <assign to="MailboxPath" from="/ProcessData/inMailbox/text()"/>
        <assign to="MessageExtractable">YES</assign>
        <assign to="OrderBy">MessageId</assign>
        <assign to="." from="*"/>
      </output>
      <input message="inmsg">
        <assign to="." from="*"/>
      </input>
    </operation>
    <sequence name="grabMessage">
      <choice name="moreMessages">
        <select>
          <case ref="atLeastOneMessage" activity="extractMessage"/>
        </select>
        <sequence name="extractMessage">
          <!-- the following assign assumes that all the files will share a naming convention
               and that there will be an underscore in the file name. Adjust as needed. There
               may be more elegant ways to do this.
          -->
          <assign to="zipFilePrefix" from="substring-before(/ProcessData/Message[1]/MessageName, '_')"/>
          <operation name="Mailbox Extract Begin Service">
            <participant name="MailboxExtractBegin"/>
            <output message="MailboxExtractBeginServiceTypeInputMessage">
              <assign to="CommitNow">NO</assign>
              <assign to="MessageId" from="/ProcessData/Message[1]/MessageId/text()"/>
              <assign to="." from="*"/>
            </output>
            <input message="inmsg">
              <assign to="." from="*"/>
            </input>
          </operation>
          <assign to="savedCount" from="number(/ProcessData/savedCount/text() + 1)"/>
          <operation>
            <participant name="MailboxExtractCommit"/>
            <output message="MailboxExtractCommitServiceTypeInputMessage">
              <assign to="MessageId" from="/ProcessData/Message[1]/MessageId/text()"/>
              <assign to="." from="*"/>
            </output>
            <input message="inmsg">
              <assign to="." from="*"/>
            </input>
          </operation>
          <!-- if the 'to' value of the next Assign changes, change the Java code, too -->
          <assign to="currentNode" from="concat(/ProcessData/savedPrefix/text(), /ProcessData/savedCount/text())"/>
          <operation name="JavaTask Service">
            <participant name="dynamic_JavaTask"/>
            <output message="JavaTaskInputMessage">
              <assign to="srcLocationMode">inline</assign>
              <assign to="javaSrc">import com.sterlingcommerce.woodstock.workflow.Document;
                final String ROOT = "/ProcessData/";
                Document doc = wfc.getPrimaryDocument();
                String nodeName = (String) wfc.getWFContent("currentNode");
                String fileName = (String) wfc.getWFContent("MessageName");
                String xpathLocation = ROOT + nodeName;
                wfc.addWFContent(xpathLocation, doc);
                wfc.addWFContent(xpathLocation + "/@filename", fileName);
                return "OK";
                </assign>
              <assign to="." from="*"/>
            </output>
            <input message="inmsg">
              <assign to="." from="*"/>
            </input>
          </operation>
          <operation name="Release Service">
            <participant name="ReleaseService"/>
            <output message="ReleaseServiceTypeInputMessage">
              <assign to="TARGET">/ProcessData/Message[1]</assign>
              <assign to="." from="*"/>
            </output>
            <input message="inmsg">
              <assign to="." from="*"/>
            </input>
          </operation>
          <operation name="Release Service">
            <participant name="ReleaseService"/>
            <output message="ReleaseServiceTypeInputMessage">
              <assign to="TARGET">/ProcessData/*[name() = 'MessageId'
              or name() = 'PrimaryDocument'
              or name() = 'DocumentId'  or name() = 'CreateDateTime'
              or name() = 'MessageName' or name() = 'MailboxPath'
              or name() = 'ContentType' or name() = 'MessageSize'
              or name() = 'TRACKINGID'  or name() = 'ExtractableCount'
              or name() = 'ExtractableUntil']</assign>
              <assign to="." from="*"/>
            </output>
            <input message="inmsg">
              <assign to="." from="*"/>
            </input>
          </operation>
          <onFault>
            <operation name="Mailbox Extract Abort Service">
              <participant name="MailboxExtractAbort"/>
              <output message="MailboxExtractAbortServiceTypeInputMessage">
                <assign to="." from="*"/>
                <assign to="MessageId" from="/ProcessData/Message[1]/MessageId/text()"/>
              </output>
              <input message="inmsg">
                <assign to="." from="*"/>
              </input>
            </operation>
          </onFault>
          <repeat name="Repeat" ref="grabMessage"/>
        </sequence>
      </choice>
    </sequence>
    <sequence name="compression">
      <choice name="filesToCompress">
        <select>
          <case ref="atLeastOneSavedNode" activity="compressThoseFiles"/>
        </select>
        <sequence name="compressThoseFiles">
          <operation name="Compress">
            <participant name="CompressionService"/>
            <output message="outputMessage">
              <assign to="." from="*"/>
              <assign to="compression_action">compress</assign>
              <assign to="compression_level">9</assign>
              <assign to="compression_type">Deflate</assign>
              <assign to="doc_to_compress">All</assign>
            </output>
            <input message="inputMessage">
              <assign to="." from="*"/>
            </input>
          </operation>
          <operation name="Delete from Mailbox">
            <participant name="MailboxDelete"/>
            <output message="MailboxDeleteServiceTypeInputMessage">
              <assign to="MailboxPath" from="/ProcessData/inMailbox/text()"/>
              <assign to="MessageExtractable">NO</assign>
              <assign to="." from="*"/>
            </output>
            <input message="inmsg">
              <assign to="." from="*"/>
            </input>
          </operation>
          <operation>
            <participant name="TimestampUtilService"/>
            <output message="xout">
              <assign to="action" from="'format'"/>
              <assign to="baseTime" from="'now'"/>
              <assign to="format">yyyyMMdd_hhmmssS</assign>
            </output>
            <input message="in">
              <assign to="." from="*"/>
            </input>
          </operation>
          <operation name="Mailbox Add Service">
            <participant name="MailboxAdd"/>
            <output message="AddRequest">
              <assign to="MailboxPath" from="/ProcessData/outMailbox/text()"/>
              <assign to="ContentType">application/zip</assign>
              <assign to="Extractable">YES</assign>
              <assign to="MessageName" from="concat(/ProcessData/zipFilePrefix/text(), '_', /ProcessData/formattedTime/text(), '.zip')"/>
              <assign to="." from="*"/>
            </output>
            <input message="inmsg">
              <assign to="AddResults" from="*"/>
            </input>
          </operation>
          <operation name="Release Service">
            <participant name="ReleaseService"/>
            <output message="ReleaseServiceTypeInputMessage">
              <assign to="TARGET">/ProcessData/*[name() = 'zipFilePrefix'
                or name() = 'PrimaryDocument'
                or starts-with(name(), 'format')
                or starts-with(name(), savedPrefix/text())
                or name() = 'Message']</assign>
              <assign to="." from="*"/>
            </output>
            <input message="inmsg">
              <assign to="." from="*"/>
            </input>
          </operation>
          <operation name="Release Service">
            <participant name="ReleaseService"/>
            <output message="ReleaseServiceTypeInputMessage">
              <assign to="TARGET">/ProcessData/*[starts-with(name(), 'saved')]</assign>
              <assign to="." from="*"/>
            </output>
            <input message="inmsg">
              <assign to="." from="*"/>
            </input>
          </operation>
        </sequence>
      </choice>
    </sequence>
  </sequence>
</process>

Note that his business process uses a Java Task Service. It's there to place the file nodes into the ProcessData DOM tree and give each node a unique value:

<?xml version="1.0" encoding="UTF-8"?>
<ProcessData>
  <inMailbox>/compress/inbox</inMailbox>
  <outMailbox>/compress/outbox</outMailbox>
  <savedPrefix>Document</savedPrefix>
  <savedCount>3</savedCount>
  <zipFilePrefix>acme850</zipFilePrefix>
  <currentNode>Document3</currentNode>
  <Document1 SCIObjectID="90580215796755485node1" filename="acme850.edi"/>
  <Document2 SCIObjectID="11589415796755485node1" filename="acme850_2.edi"/>
  <Document3 SCIObjectID="79598615796755485node1" filename="acme850_3.edi"/>
  <PrimaryDocument SCIObjectID="23607615796755485node1"/>
</ProcessData>

The compression service can then place all the files into a single zip file.

This approach will work best when there will be a limited number of files to be compressed; if there are hundreds of files, this implementation may prove unwieldy.  Too, it it will be advisable to test thoroughly before using this business process in a Production environment.

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS3JSW","label":"IBM Sterling B2B Integrator"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"","Edition":"","Line of Business":{"code":"LOB59","label":"Sustainability Software"}}]

UID

ibm11121379