Implementing a database-based caching pattern in WebSphere Enterprise Service Bus

Learn how to create a dynamic database-based caching pattern in WebSphere ESB by extending the packaged primitives with an additional set of mediation primitives. The article describes the architecture of the caching pattern and shows you how to implement your own primitives.

Thomas Bohn (tbohn@de.ibm.com), Certified IT Specialist, IBM Software Services for WebSphere, IBM

Photo of Thomas BohnThomas Bohn is a Certified IT Specialist with the IBM Software Services for WebSphere team in Germany. He has worked on this team for 12 years, and his current focus is architecture for BPM and software integration projects. His product specialties include WebSphere Application Server, WebSphere ESB, and WebSphere Process Server.



Susann Fritzsche (susann.fritzsche@de.ibm.com), IT Specialist, IBM Software Services for WebSphere, IBM

Photo of Susann FritzscheSusann Fritzsche is an IT Specialist with the IBM Software Services for WebSphere team in Germany. She has worked on this team since joining IBM three years ago, and her current specialties include Business Process Management and WebSphere ESB.



21 November 2012

Also available in Chinese Spanish

Introduction

According to commonly accepted SOA architecture, the integration layer in an SOA infrastructure provides stateless services to other components. So in any SOA implementation, data flows through the integration layer between back-end and front-end systems. Depending on the scenario, many of the requests repeatedly access the same information. Therefore caching data in the integration layer can be an effective way to limit resource consumption and improve response times.

In a recent project the authors were faced with a requirement that extended that simple caching pattern. The integration layer provided services for a portal-based front-end to enable user to work with large lists of objects by using functions like scrolling, filtering, and sorting while having pages of data presented to them. Since the data came from a relatively slow back end, caching it made lots of sense. But the portal could not host all of that information , because its per-user granularity would create a huge amount of cached data with a significant performance impact.

Therefore we implemented a data cache in the integration layer that supported a per-user granularity. In order to implement functions to scroll through, sort, and filter lists of objects that are retrieved once from the back end, we had to implement the cache in a stateful, per-user manner, which is an unusual pattern according to current SOA architecture standards.

This article describes this pattern, the solution based on it, and the base technology provided by IBM® WebSphere® Enterprise Service Bus (hereafter called WebSphere ESB). The article describes a smart, dynamic, and powerful database-based caching pattern that, when combined with IBM DB2, enables WebSphere ESB to take workload off of any back-end system, particularly for handling lists of elements. The added value of this caching solution lies in its awareness of the semantics of cached data structures in order to provide filtering, paging, and sorting mechanisms for the caller. The caching pattern is based on the ability to extend the pre-packaged WebSphere ESB mediation primitives, and the article includes an overview of how to develop your own mediation primitives.

To benefit from this article, your should have integration experience using WebSphere ESB, and a basic understanding of IBM DB2® and its pureXML features that support storing and querying XML data with the SQL/XML and XQuery languages.

Overview and purpose of a database-based caching pattern for WebSphere ESB

The database-based caching pattern for WebSphere ESB (which is called the Stateful Business Object List Cache) supports smart, stateful caching and repeated retrieval of service response messages that contain a list of business objects in order to:

  • Enable browser-based front-end systems to retrieve sets of data pagewise, filtered and sorted to provide more convenient access to end users
  • Minimize the number of requests sent to back-end services and systems

In order to work, the functions on WebSphere ESB must receive information from the caller specifying the criteria to retrieve data. Any call to the Stateful Business Object List Cache must therefore contain information for the retrieval of the cached business object list, and those criteria are passed along as part of the request message by the requester of the service. Figure 1 shows the object ResultRetrieveCriteria, and Table 1 below it describes its attributes in detail:

Figure 1. ResultRetrieveCriteria business object
ResultRetrieveCriteria business object
Table 1. ResultRetrieveCriteria attributes
Attribute nameDescription
pagingStartIndexDefines the start index in the list for paging
pagingMaxResultDefines how many objects of the list starting at pagingStartIndex will be returned
sortFieldDefines the field for that the list will be sorted. The XPath expression has to be relative to the list root element in the response object.
sortASCDefines whether the list will be sorted ascending (true) or descending (false)
searchListNameDefines the name of the list in the response object to which the search is applied.
searchFieldsDefines list of attributes (nameXPath, value) as the search criteria. The XPath expression for the attribute name must be relative to the list root element in the response object.
searchCombinationTypeDefines the search combination type. ALL search criteria are linked by AND or OR.

The cache is used with graphical user interfaces (GUIs) such as portals to display lists of objects and provide functions such as sorting, searching, and paging. The WebSphere ESB side caching components must maintain a logical session with the front-end system in order to match cached data with the user executing the query. To support this per-user granularity, every cached element is identified by two keys that must be unique when combined:

Cache Session Key
This key is for the logical session that an entity, usually a user, has with the asset. The value of the key should be the same across multiple cache entries created for the same user. The HTTP Session ID of a front-end system may be used for the cache session key.
Cache Entry Key
This key uniquely identifies the cached element from a business data point of view. It can include multiple elements of the request business object that in combination must uniquely identify the response business object instance.

The caching pattern uses the Cache Session Key and the Cache Entry Key to store the list of business objects as serialized XML in the database. The caching pattern uses IBM DB2 and its pureXML functions to store cached data and implement paging, sorting, and filtering. The caching pattern translates requests for cached data into SQL statements that are executed in the database. Logically, the caching pattern acts as a façade for database functions and translates them into service calls. For more information about the DB2 PureXML feature, see Resources at the bottom of the article.

Caching flow

This section explains what the caching flow looks like. Figure 2 shows the execution phases of the Stateful Business Object List Cache. The primitives used in this caching pattern are described in more detail in the Caching primitives section below.

Figure 2. Caching flow
Caching flow
Path 1
The StatefullBOListCacheGet primitive that is placed in the request flow is executed. It extracts the cache session key and calculates the cache entry, and then uses both of them to check if a corresponding entry exists in the database. If a cache entry is found, the primitive executes Path 4. If no cache entry is found, the primitive executes Path 2.
Path 2 -- Cache Fail
The StatefullBOListCacheGet primitive forwards the cache session key, the cache entry key, and other data to the StatefullBOListCachePut primitive. The StatefullBOListCacheGet primitive calls the cache fail terminal. You must put the ResultRetrieveCriteria object contained in the request message into a location accessible to the StatefullBOListCachePut primitive. The suggested location is within the correlation context. The request message is then sent to the back-end business service for processing. The control flow is continuous with Path 3.
Path 3
The back-end business service responds with a response message sent to the StatefullBOListCachePut primitive. The primitive retrieves the data provided by the StatefullBOListCacheGet primitive in order to insert the response message into the database. The list of data objects is serialized and stored in a database table column of type XML. The StatefullBOListCachePut primitive then retrieves the ResultRetrieveCriteria object in order to generate a database query that will apply the filter, sorting, or searching functions that the requester wants to be executed on the list of business objects contained in the response message. The StatefullBOListCachePut primitive executes the database query and sends its response back to the requester.
Path 4 -- Cache Hit
The StatefullBOListCacheGet primitive retrieves the ResultRetrieveCriteria object in the request in order to generate a database query that will apply the filter, sorting, or searching functions that the requester wants to be executed on the list of business objects contained in the response message. The StatefullBOListCachePut primitive executes the database query and sends its response back to the requester, avoiding a call to the back-end business service.

Caching primitives

This section describes the primitives that the Stateful Business Object List Cache uses.

StatefullBOListCacheGET

This primitive is part of the mediation request flow.

The cacheFail terminal, which is called if no cache entry for the sessionID and key exists, returns the request message so that the original message can be passed to the back end. The cacheHit terminal, which is called if a cache entry for the sessionID and key exists, returns the response message to which the searching, paging, and sorting mechanism has already been applied. This response should be passed back to the caller without making further calls to the back-end system. Table 2 shows the primitive properties in the Details section of the Properties view in WebSphere Integration Developer:

Table 2. StatefullBOListCacheGET primitive properties
PropertyDescription
Cache DB data source JNDI nameJNDI name of data source
XPath to unique ID that identifies the client's session with the cacheThis ID is a key for the logical session that an entity (usually a user) has with the asset. In order to clear out all activities of a user, the value of the key should be the same across multiple cache entries created for the same user. For example, the HTTP Session ID of a front-end system may be used for the cache session key.
XPath to object of type ResultRetrieveCriteria that specifies how the response should be sent back.The XPath expression should point to the ResultRetrieveCriteria object within the operation input object.
XPath to integer attribute in the response message that holds the count of result objects in the BO list.The XPath expression must point to an integer attribute that holds the count of result objects in the BO list that meet the searching, paging, and sorting criteria.
XPath to the response message list root and attribute that is subject to caching.The XPath expression has to point to the attribute that is the list root element. The path should start at the operation response object level.
Cache keysUniquely identify the cached element from a business data point of view. It can include multiple elements of the request business object that in combination must uniquely identify the response business object instance.

This primitive is part of the mediation response flow. It uses the defined cache key and the unique session identifier from the GET primitive to store the serialized response message from the backend system in the cache database. Furthermore it executes a search, paging and sorting on the stored list of objects for the ResultRetrieveCriteria defined as part of the request input object. Table 3 shows the primitive properties in the Details section of the Properties view in WebSphere Integration Developer:

Table 3. StatefullBOListCachePUT primitive properties
PropertyDescription
Cache DB datasource JNDI nameJNDI name of data source
XPath to object of type ResultRetrieveCriteria that specifies how the response should be sent back.XPath expression should point to the ResultRetrieveCriteria object.

The out terminal of the PUT primitive returns the response object containing the list of objects that meet the ResultRetrieveCriteria. This response object can be passed to the input response node.

This primitive deletes a cache database entry for a given key and session ID. It is placed in the request flow in front of the GET primitive. The front-end system triggers the execution by setting a flag in the operation input object to assure that the request is sent to the back-end system. Table 4 shows the primitive properties that are part of the Details section of the Properties view in WebSphere Integration Developer.

Table 4. StatefullBOListCache INVALIDATE WITH KEY AND SESSION
PropertyDescription
Cache DB datasource JNDI nameJNDI name of data source.
XPath to unique ID that identifies the client's session with the cacheThis ID is a key for the logical session that an entity (usually a user) has with the asset. In order to clear out all activities of a user, the value of the key should be the same across multiple cache entries created for the same user. For example, the HTTP Session ID of a front-end system may be used for the cache session key.
Cache keysUniquely identify the cached element from a business data point of view. It can include multiple elements of the request business object that in combination must uniquely identify the response business object instance.

This primitive deletes all cache entries that belong to a specific session ID and is called as a separate service by the front-end system when the user session expires. Table 5 shows the primitive properties in the Details section of the Properties view in WebSphere Integration Developer.

Table 5. StatefullBOListCache INVALIDATE FOR ENTIRE SESSION
PropertyDescription
Cache DB datasource JNDI nameJNDI name of data source.
XPath to unique ID that identifies the client's session with the cacheUniquely identify the cached element from a business data point of view. It can include multiple elements of the request business object that in combination must uniquely identify the response business object instance.

This primitive invalidates the whole cache database and can be called (once per night for example) by a scheduler service to clean up the cache. Table 5 shows the primitive properties in the Details section of the Properties view in WebSphere Integration Developer.

Table 6. StatefullBOListCache INVALDIATE FOR COMPLETE CACHE DB
PropertyDescription
Cache DB datasource JNDI nameJNDI name of data source.

Overview of building your own mediation primitives

The StatefullBOList Cache uses its own mediation primitives. Here is an overview of how to build your own mediation primitives:

  1. Build the plug-in project. It defines the visual aspect, and specifies which properties are available in the Details section of your primitive.
  2. Build the Java™ project that contains the logic and implementation of your primitive.
  3. Deploy the plug-in project as an Eclipse plug-in in WebSphere Integration Developer, so that it appears on the Mediation Flow Editor palette.
  4. Deploy the Java project and make the implementation available to the runtime.
  5. Test your custom mediation primitive.

For a detailed description of each step, see the first two items below under Resources.

Conclusion

This article introduced a smart, dynamic, and powerful database-based caching pattern in WebSphere Enterprise Service Bus and outlined the pattern and its architecture. The caching pattern is based on a set of mediation primitives. The article then gave a brief overview of how to build your own mediation primitives.

Acknowledgements

The authors would like to thank Shili Yang of IBM Software Services for WebSphere for her help in reviewing this article.

Resources

  • WebSphere ESB resources
  • Resources for the DB2 pureXML feature
    • 15 best practices for pureXML performance in DB2
      The DB2 pureXML feature enables you to store and query XML in its inherent hierarchical format using either SQL/XML or XQuery, and it also supports sophisticated XML indexing and XML schema validation. This very popular developerWorks article provides XML-specific DB2 performance tips.
    • Get off to a fast start with DB2 V9 pureXML, Part 3: Query DB2 XML data with SQL
      This developerWorks article shows you how to use DB2 pureXML features to query data stored in XML columns using SQL and SQL/XML.
    • Updating XML in DB2 V9.5
      DB2 V9.5 introduced the XQuery Update Facility, an extension to XQuery that lets you modify, insert, or delete individual elements and attributes within an XML document, to make updating XML data easier improve performance. This developerWorks article describes the new XML update functionality, presents examples of XML update operations, and shows you how to avoid common pitfalls.
    • Use of updating expressions in a transform expression
      Explanation of DB2 XQuery updating expressions from the DB2 information center.
  • WebSphere resources
    • developerWorks WebSphere developer resources
      Technical information and resources for developers who use WebSphere products. developerWorks WebSphere provides product downloads, how-to information, support resources, and a free technical library of more than 2000 technical articles, tutorials, best practices, IBM Redbooks, and online product manuals.
    • developerWorks WebSphere application integration developer resources
      How-to articles, downloads, tutorials, education, product info, and other resources to help you build WebSphere application integration and business integration solutions.
    • Most popular WebSphere trial downloads
      No-charge trial downloads for key WebSphere products.
    • WebSphere forums
      Product-specific forums where you can get answers to your technical questions and share your expertise with other WebSphere users.
    • WebSphere on-demand demos
      Download and watch these self-running demos, and learn how WebSphere products and technologies can help your company respond to the rapidly changing and increasingly complex business environment.
    • WebSphere-related articles on developerWorks
      Over 3000 edited and categorized articles on WebSphere and related technologies by top practitioners and consultants inside and outside IBM. Search for what you need.
    • developerWorks WebSphere weekly newsletter
      The developerWorks newsletter gives you the latest articles and information only on those topics that interest you. In addition to WebSphere, you can select from Java, Linux, Open source, Rational, SOA, Web services, and other topics. Subscribe now and design your custom mailing.
    • WebSphere-related books from IBM Press
      Convenient online ordering through Barnes & Noble.
    • WebSphere-related events
      Conferences, trade shows, Webcasts, and other events around the world of interest to WebSphere developers.
  • developerWorks resources
    • Trial downloads for IBM software products
      No-charge trial downloads for selected IBM® DB2®, Lotus®, Rational®, Tivoli®, and WebSphere® products.
    • developerWorks business process management developer resources
      BPM how-to articles, downloads, tutorials, education, product info, and other resources to help you model, assemble, deploy, and manage business processes.
    • developerWorks blogs
      Join a conversation with developerWorks users and authors, and IBM editors and developers.
    • developerWorks tech briefings
      Free technical sessions by IBM experts to accelerate your learning curve and help you succeed in your most challenging software projects. Sessions range from one-hour virtual briefings to half-day and full-day live sessions in cities worldwide.
    • developerWorks podcasts
      Listen to interesting and offbeat interviews and discussions with software innovators.
    • developerWorks on Twitter
      Check out recent Twitter messages and URLs.
    • IBM Education Assistant
      A collection of multimedia educational modules that will help you better understand IBM software products and use them more effectively to meet your business requirements.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=846513
ArticleTitle=Implementing a database-based caching pattern in WebSphere Enterprise Service Bus
publish-date=11212012