Loading IBM Integration Bus configuration data from a file using the FileRead node and shared variables

This article shows you how to read IBM Integration Bus V9 configuration data from a file and load it to memory for faster access in message flows. Topics include the purpose of loading data to memory, solution design, and sample implementation.

Vrunda R. Negandhi (vrunda_negandhi@in.ibm.com), Integration Specialist, Business Analytics Solution Development, IBM

Photo of Vrunda R. NegandhiVrunda R. Negandhi is an Integration Specialist with the Business Analytics Solution Developmen team in India. She has eight years of experience with J2EE and integration products such as IBM Integration Bus, WebSphere Message Broker, and WebSphere DataPower Appliances. You can contact Vrunda at vrunda_negandhi@in.ibm.com.



Sumitha M. Prasannam (sumithmp@in.ibm.com), Integration Specialist, Business Analytics Solution Development, IBM

Photo of Sumitha M. PrasannamSumitha M. Prasannam is an Integration Specialist with the Business Analytics Solution Developmen team in India. She has eight years of experience with integration products such as IBM Integration Bus, WebSphere Message Broker, WebSphere MQ, and WebSphere DataPower Appliances. You can contact Sumitha at sumithmp@in.ibm.com.



27 December 2013

Why load configuration data into memory?

In some application scenarios, it is better to use a configuration file with static data and load it into memory only once, when the flow starts, instead of loading it before every flow execution. For example:

  • When you want to use an external file to control the sequence in which applications and message flows are invoked
  • When you want to store static, environment-specific information outside of IBM Integration Bus, so it can be managed by people with limited product and application knowledge.
  • When you want to control log levels and log file names through a configuration file
  • When you want to store multi-language statements and error messages in an external file

In the above scenarios, keeping the configuration files outside IBM Integration Bus frees you from editing the message flow whenever there is a change to the data. Also, reading a configuration file during every run of a message flow is time-consuming, and loading them into memory once means fewer I/O operations, faster data access, and better performance.

Design

  • The FileRead Node is used to read the configuration file from within a message flow.
  • Use shared variables to load configuration data to an in-memory cache in the message flow. The configuration data is loaded when the first message passes through the flow after each broker startup. Shared variables are long-lasting and can persist during the lifetime of the execution group process, the lifetime of the flow or node, or the lifetime of the SQL node that declares the variable, whichever is the shortest.
  • You can use shared variables anywhere in the scope of the message flow to access the in-memory data. They are visible to all messages passing through the flow. Whenever there is a change to the configuration data, you can edit the configuration file and restart the message flow to implement the change.

Sample implementation

Here is an example that involves storing web service credentials in a configuration file, loading it into memory during the first run of the message flow, and then accessing it during all subsequent transactions in the message flow. The configuration file name is credentials.properties, and it contains the Username admin and the Password adminpwd. You use the FileRead Node to read the configuration data:

  1. Create a DFDL to define the structure of the configuration file: Select File => New => Message Model, aelect Record-oriented text as the Format, and click Next.
  2. Select Create a DFDL schema file using this wizard to guide you and then click Next. Enter the DFDL schema file name and message name:
  3. Click Next and enter 2 for Number of body fields. Uncheck The first record is a header and The last record is a trailer. Click Finish:
  4. In the DFDL created, edit the field names as shown below:
  5. Edit the Separator property of sequence under the Representation property to =:
  6. Here is the message flow demonstrating the loading of configuration data into memory:

BuildRequest

From the BuildRequest node before propagating to the FileRead node(out2), a check is performed to see if the configuration shared variable is loaded. If he shared variable is not available, then the file read is triggered. Usually, loading the configuration data to shared variables is performed only during the first run of the flow. Here is some sample code to trigger the loading of configuration data into memory:

FileRead Node

In the FileRead node, configure the Input directory from where the file needs to be read and the File name, as shown below:

For Message domain, select DFDL and then select the message model:

LoadToMemory

Using the code below, load the data read from the file into a shared variable:

Control then goes back to the BuildRequest Node:

BuildRequest

It executes the code below to set the credentials in the request:

Conclusion

The article explained the advantages of loading IBM Integration Bus configuration files into memory, and showed you how to do it during the first executions of a mesage flow.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=958661
ArticleTitle=Loading IBM Integration Bus configuration data from a file using the FileRead node and shared variables
publish-date=12272013