Object Store Client Adapter

The ObjectStoreClientadapter is a critical component of the Sterling B2B Integrator, allowing seamless connection with Azure Blob Storage. You can connect your Sterling B2B Integrator to Azure's powerful storage capabilities by utilising the Azure Blob Storage within the ObjectStoreClientadapter.

Azure Blob Storage helps you create data lakes for your analytics needs and provides storage to build powerful cloud-native and mobile apps. Optimize costs with tiered storage for your long-term data and flexibly scale up for high-performance computing and machine learning workloads. With this integration, the Sterling B2B Integrator can harness the virtually unlimited scalability of Azure Blob Storage, efficiently manage and store large volumes of unstructured data, access data seamlessly, and optimize costs based on specific requirements.

In Sterling B2B Integrator v6.2.0.0, you can easily perform operations such as list, get, put, and delete files, as well as make dir.

In future releases, the ObjectStoreClientadapter will be extended to connect to Google Cloud and S3 enabled Object Stores. Hence, the adapter name and configuration parameter are generic.

Note: The ObjectStoreClientadapter uses the Azure azure-storage-file-datalake library, which works with only on Azure Data Lake Storage Gen2 accounts. Hence,
  • The Azure account should be upgraded to StorageV2 and it should have the necessary permissions to read, write, and list the container configured.
  • Blob versioning is unsupported.
The following table shows Sterling B2B Integrator and Azure Blob Storage terminology:
Sterling B2B Integrator Azure Blob Storage
Bucket Container
File / Document Blob
Folder Directory
sessiontoken SAS token
Secret key/Account Key Account Key
The ObjectStoreClientadapter has the following major features:
  • Commands are scriptable through BPML
  • Accessible through the following services in a business process:
    • Object Store Client Begin Session service
    • Object Store Client LIST service
    • Object Store Client MKDIR service
    • Object Store Client PUT service
    • Object Store Client GET service
    • Object Store Client DELETE service
    • Object Store Client End Session service
The following table provides an overview of the ObjectStoreClientadapter:
Category Description
System name ObjectStoreClientadapter
Graphical Process Modeler (GPM) category None
Description Sends Object Store (currently Azure) requests to trading partners.
Business usage Use this adapter to send Object Store requests to perform activities such as to get or put files into a directory on Azure Blob Storage.
Usage Example A business process is executed that translates a document to send to a trading partner. After the translation, the ObjectStoreClientadapter sends the document to the trading partner.
Preconfigured? Yes, with default properties. You need to provide your specific values.
Requires third-party files? No
Platform availability All Sterling B2B Integrator supported platforms
Related services The following services are related. Configured in a business process, they initiate the ObjectStoreClientadapter to perform their services:
  • Object Store Client Begin Session service
  • Object Store Client LIST service
  • Object Store Client MKDIR service
  • Object Store Client PUT service
  • Object Store Client GET service
  • Object Store Client DELETE service
  • Object Store Client End Session service
Application requirements The ObjectStoreClientadapter should be configured with the Azure Blob Storage account, credentials, and bucket for trading partner.
Initiates business processes This adapter does not initiate business processes.
Invocation This adapter is not invoked from a business process.
Business process context considerations Business processes using Object Store Client services should not be marked as Auto Resume. These services require an established session which will no longer exist after a restart.
Returned status values None
Restrictions All work bound within an Object Store Client Begin Session service and an Object Store Client End Session service must be within the same business process.
Persistence level N/A
Environment Supported Environments are:
  • ASI nodes
  • AC nodes
Note:
  • The ObjectStoreClientadapter does not support all adapter container node environment. Instead, use specific nodes as a target environment.
  • When the ObjectStoreClientadapter is hosted on an AC node, then JMS MQ should be configured.
Testing considerations To test this adapter, run the ObjectStoreClientDemoAllServices business process and verify that it completes successfully. For more information about the ObjectStoreClientDemoAllServices business process, see the Business Process Example of All Object Store Client Services. Debug information for this adapter is located at: Operations > System > Logs > Log.ObjectStoreClient.

Implementing the ObjectStoreClientadapter

To implement the ObjectStoreClientadapter:
  1. Create an ObjectStoreClientadapter configuration. See Sterling B2B Integrator Services and Adapters.
  2. Configure the ObjectStoreClientadapter. See Configuring the ObjectStoreClientadapter.

Configuring the ObjectStoreClientadapter

To configure the ObjectStoreClientadapter:
  1. Select Deployment > Services > Configuration.
  2. Search for ObjectStoreClientadapter or select it from the list.
  3. Click Edit.
  4. Specify field settings:
    Field Description
    Name Unique and meaningful name for the adapter configuration. Required.
    Description Description of adapter. Required.
    Select a Group
    Select one of the options:
    • None – You do not want to include this configuration in a group at this time.
    • Create New Group – You can enter a name for a new group in this field, which will then be created along with this configuration.
    • Select Group – If you have already created one or more groups for this adapter type, they are displayed in the list. Select group from the list.
      Note: For more information about groups, see Sterling B2B Integrator Services and Adapters.
    Object Store Type

    Select Object Store Type: Azure
    EndPoint URL EndPoint URL of the Azure Blob Storage
    Region Region of the Azure Blob Storage container
    Access Key Not used for Azure Blob storage.
    Secret Key or Account Key Account Key for Azure Blob Storage account
    Bucket Name

    Bucket Name. Optional if it is present in begin session or individual services.

    See Sterling B2B Integrator and Azure Blob Storage terminology for nomenclature.
    Minimum Number of Threads A tuning parameter that indicates the range of threads available for handling events to improve performance. For more information, see Performance and Tuning. Must be less than or equal to the maximum number of threads value. Required. The default value is three.
    Maximum Number of Threads A tuning parameter that indicates the range of threads available for handling events to improve performance. For more information, see Performance and Tuning. Must be greater than or equal to the minimum number of threads value. Required. The default value is six.
    Connection Retries The number of times the adapter tries to connect to the Azure Blob Storage service when there is a connection error. Optional. The default value is three.
    Connection Timeout Amount of time in milliseconds after which the connection terminates in case of a connection error and the number of connections retries is attempted. Optional. The default value is 30000 milliseconds.
    Read Timeout Time out value in milliseconds. Optional. The default value is 30000 milliseconds.
    Proxy Server

    Whether to use a proxy server. Valid values are:

    • Yes – Proxy server is used
    • No – No proxy server is used
    Required.
    Proxy hostname A valid proxy server hostname or IP address to use as a gateway. Required if proxy server is set to Yes.
    Proxy port Listen port number of the proxy server. Required if proxy server is set to Yes.
    Use proxy basic authentication

    Whether the proxy server requires basic authentication. Valid values are:

    • Yes – Proxy basic authentication will be used
    • No – Proxy basic authentication will not be used
    Required if proxy server is set to Yes.
    Proxy username Username to connect to the proxy server. Required if use proxy basic authentication is set to Yes.
    Proxy password Password to connect to the proxy server. Required if use proxy basic authentication is set to Yes.
    Use SSL Not supported in the Sterling B2B Integrator v6.2.0.0.
    Document Storage Type

    Select Document Storage Type:

    • File System
    • Database
    • System Default
    • Object Store [available on OCP]
    Note: This document storage type option will be used from BP and not from the adapter configuration.
Note: For all services, the filepattern parameter:
  • Object Store prefixes (similar to folders) are included when matching the filepattern.
  • The filepattern must contain a valid regular expression supported by Java 8.
  • The following are the example values of filepattern:
    Assignment Result
    <assign to="filepattern">.+</assign> All Objects
    <assign to="filepattern">.*log.*</assign> All objects which contain "log"
    <assign to="filepattern">fvt.+</assign> All objects whose name starts with "fvt"
    <assign to="filepattern">.*.xml</assign> All objects whose extension is ".xml"

Configuring the objectstoreclient.properties file

The objectstoreclient.properties file in the properties directory provides settings for the ObjectStoreClientadapter. These properties can also be set through customer_override. The key for the file is objectstoreclient. For more information, see Customizing Property Files.

Default Values for the Property

  • DelayWaitingOnIO. The default value is 10.
  • copyBufferSize. The default value is 32768.
  • disablePersistForDocTracking. The default value is false.
  • idleTimeout. The default value is 20.
  • maxTotalConnections. The default value is three.

Configuring the objectstoreclient.properties file

  • Locate the objectstoreclient.properties.in file in the properties directory where you installed Sterling B2B Integrator.
  • Edit the objectstoreclient.properties.in file.
  • Configure the properties.

The properties are listed in the following table:

DelayWaitingOnIO Specifies the number of seconds to wait for the data transfer to complete before going into the WAITING_ON_IO state.
copyBufferSize Buffer size in GET and PUT services.
disablePersistForDocTracking Property that disables all persist calls in the Object Store client get service. If this is true, the doc tracking information will not persist.
Valid values:
  • True: Doc information will not be persisted.
  • False: Doc information will be persisted (default).
idleTimeout Idle timeout in seconds for connections in a connection pool.
maxTotalConnections The maximum number of connections in a pool.
Note: If there are a number of files or large files for PUT and GET services, then make sure to increase the DelayWaitingOnIO property.

Business Process Example of All Object Store Client Services

To help you get started using the ObjectStoreClientadapter and Object Store Server adapter, Sterling B2B Integrator includes a demo that provides an example of all the services. The demo transfers a file from the ObjectStoreClientadapter to Azure. The following graphic shows the business process model of ObjectStoreClientDemoAllServices.

The BPML code associated with ObjectStoreClientDemoAllServices is shown below:

To implement the ObjectStoreClientDemoAllServices, replace the dummy values with the actual values in the following BP:
Note: The parameters can be selected according to the use case, not all the parameters are mandatory. The detailed description of parameters is mentioned in the Parameter Description section of individual Object Store services.
Note: You can obscure the secretkey or accountkey using the Obscure Data - Process Data Values service and use it in the ObjectStoreclientbeginservice business process.

The following are the instructions for running this business process:

Run ObjectStoreClientDemoAllServices services.
  1. Go to Business Process > Manager.
  2. Search for ObjectStoreClientDemoAllServices.
  3. Click Execution Manager > Execute.
  4. Click Go.
<process name="ObjectStoreClientDemoAllServices">
<sequence>

<!-- Get obscured password -->
<operation name="Obscure Password">
    <participant name="ObjectStoreClientObscureParameter"/>
    <output message="outmsg">
        <assign to="." from="*"></assign>
    </output>
    <input message="inmsg">
        <assign to="." from="*"></assign>      
    </input>
</operation>

<operation name="Object Store Client Begin Session Service">
    <participant name="ObjectStoreClientBeginSession"/>
    <output message="ObjectStoreClientBeginSessionTypeInputMessage">
	  <assign to="objectstoreclientadapter">dummyobjectstoreclientadapter</assign>      
	  <assign to="accountname">dummyaccountname</assign>                              
	  <!-- This is plain text secret key -->
	  <assign to="secretkey">dummysecretkey</assign>         
	  <!-- OR -->
	  <!-- To retrieve the obscured password from process data and unobscure it using the revealObscured XPath function use -->
	  <assign to="secretkey" from="revealObscured(secretpass)" ></assign>                                  
	  <assign to="bucketname">dummybucketname</assign>	                                 
	  <assign to="connectiontimeout">dummyconnectiontimeout</assign>         
	  <assign to="readtimeout">dummyreadtimeout</assign>                            
	  <assign to="." from="*"></assign>
    </output>
    <input message="inmsg">
        <assign to="ObjectStoreClientBeginSessionServiceResults" from="*"></assign>
    </input>
</operation>

<operation name="Object Store Client MKDIR Service">
	<participant name="ObjectStoreClientMkdir"/>
	<output message="ObjectStoreClientMkdirServiceTypeInputMessage">
	  <assign to="session_token" 
                 from="ObjectStoreClientBeginSessionServiceResults/session_token/text()">
	  </assign>
	  <assign to="bucketname">dummybucketname</assign>
	  <assign to="foldername">dummyfoldername</assign>   
	  <assign to="." from="*"></assign>              
	</output>
	<input message="inmsg">
	  <assign to="ObjectStoreClientMkdirServiceResults" from="*"></assign>
	</input>
</operation>

<operation name="Object Store Client PUT Service">
    <participant name="ObjectStoreClientPut"/>
    <output message="ObjectStoreClientPutServiceTypeInputMessage">
	  <assign to="session_token" 
            from="ObjectStoreClientBeginSessionServiceResults/session_token/text()">
	  </assign>      	
	  <assign to="bucketname">dummybucketname</assign>      
	  <assign to="documentlist">dummydocumentlist</assign>		
	  <assign to="documentid">dummydocumentid</assign>                   
	  <assign to="foldername">dummyfoldername</assign>                      
	  <assign to="remotefilename">dummyremotefilename </assign>            
	  <assign to="sourcefilelist">dummysourcefilelist1, dummysourcefilelist2</assign>     
	  <assign to="filepattern">dummyfilepattern</assign>                   	
	  <assign to="." from="*"></assign> 
    </output>
    <input message="inmsg">
	  <assign to="ObjectStoreClientPutServiceResults" from="*"></assign>
    </input>
</operation>

<operation name="Object Store Client Get Service">
	<participant name="ObjectStoreClientGet"/>
	<output message="ObjectStoreClientGetServiceTypeInputMessage">
	  <assign to="session_token" 
                    from="ObjectStoreClientBeginSessionServiceResults/session_token/text()">\
	  </assign>
	  <assign to="foldername">dummyfoldername</assign>                   
	  <assign to="filepattern">dummyfilepattern</assign>              
	  <assign to="remotefilename">dummyremotefilename</assign>          
	  <assign to="remotefilelist">dummyremotefilelist1, dummyremotefilelist2</assign> 
	  <assign to="."from="*"></assign>
	</output>
	<input message="inmsg">
	  <assign to="ObjectStoreClientGetServiceResults" from="*"></assign>
	</input>
</operation>

<operation name="Object Store Client List Service">
	<participant name="ObjectStoreClientList"/>
	<output message="ObjectStoreClientListServiceTypeInputMessage">
	  <assign to="session_token" 
              from="ObjectStoreClientBeginSessionServiceResults/session_token/text()">
	  </assign>
	  <assign to="bucketname">dummybucketname</assign>
	  <assign to="foldername">dummyfoldername</assign>          
	  <assign to="filepattern">dummyfilepattern</assign>    
	  <assign to="." from="*"></assign>
	</output>
	<input message="inmsg">
	  <assign to="ObjectStoreClientListServiceResults" from="*"></assign>
	</input>
</operation>

<operation name="Object Store Client DELETE Service">
	<participant name="ObjectStoreClientDelete"/>
	<output message="ObjectStoreClientDeleteServiceTypeInputMessage">
	  <assign to="session_token" 
               from="ObjectStoreClientBeginSessionServiceResults/session_token/text()">
	  </assign>
	  <assign to="bucketname">dummybucketname</assign>
	  <assign to="foldername">dummyfoldername</assign>                
	  <assign to="remotefilename">dummyremotefilename</assign>      
	  <assign to="." from="*"></assign>
	</output>
	<input message="inmsg">
	  <assign to="ObjectStoreClientDeleteServiceResults" from="*"></assign>
	</input>
</operation>
	
<operation name="Object Store Client End Session Service">
	<participant name="ObjectStoreClientEndSession"/>
	<output message="ObjectStoreClientEndSessionServiceTypeInputMessage">
	  <assign to="session_token" 
                      from="ObjectStoreClientBeginSessionServiceResults/session_token/text()">
	  </assign>
	  <assign to="." from="*">
	  </assign>	  
	  </output>
	<input message="inmsg">
	  <assign to="ObjectStoreClientEndSessionServiceResults" from="*"></assign>
	</input>
</operation>

</sequence>
</process>

<!-- To catch all onFault handling to log an error to ObjectStoreClientAdapter -->
<onFault>
<sequence name="End Session">
<operation name="Object Store Client End Session Service">
	<participant name="ObjectStoreClientEndSession"/>
    <output message="ObjectStoreClientEndSessionServiceTypeInputMessage">
	  <assign to="session_token" 
            from="ObjectStoreClientBeginSessionServiceResults/session_token/text()">
	  </assign>
	  <assign to="." from="*"></assign>
    </output>
    <input message="inmsg">
	  <assign to="ObjectStoreClientEndSessionServiceOnFaultResults" from="*"></assign>
    </input>
</operation>
</sequence>
</onFault>

The following business process uses the ObjectStoreClientadapter to get all files from one folder and put them in another folder on Object Store.

[[Insert Object Store Client Begin Session]]

<operation name="Object Store Client Get Service">
    <participant name="ObjectStoreClientGet"/>
    <output message="ObjectStoreClientGetServiceTypeInputMessage">
		<assign to="bucketname">dummybucketname</assign>
		<assign to="foldername">dummyfoldername</assign>
		<assign to="." from="*"></assign>
    </output>
    <input message="inmsg">
		<assign to="ObjectStoreClientGetServiceResults" from="documentlist"></assign>
    </input>
</operation>
 
<operation name="Object Store Client PUT Service">
    <participant name="ObjectStoreClientPut"/>
    <output message="ObjectStoreClientPutServiceTypeInputMessage">
		<assign to="bucketname">dummybucketname/dummyfoldername</assign>
		<assign to="." from="//ObjectStoreClientGetServiceResults/documentlist" append="true"></assign>
		<assign to="." from="*"></assign>
    </output>
    <input message="inmsg">
		<assign to="ObjectStoreClientPutServiceResults" from="*"></assign>
    </input>
</operation>

[[Insert Object Store Client End Session]]