Dr. Eoin Lane, Senior Solution Engineer, is the lead for harvesting and developing of application pattern from key IBM SOA engagements and driving those patterns through IBM pattern governance process to accelerate adoption. Eoin also specializes in Model Driven Development (MDD), asset based development and Reusable Asset Specification (RAS) to facilitate SOA development.

IBM Redbook on Strategic Reuse with Asset-Based Development

A good friend of mine Lee Ackerman let me know last week that a IBM have just publicly released a draft of a new Redbook, titled: "Strategic Reuse with Asset-Based Development". The book is available from: http://www.redbooks.ibm.com/redpieces/abstracts/sg247529.html?Open

The Redbook is structured as follows:

• Chapter 1. Introduction to Asset-Based Development
• Chapter 2. ZYX Electronics case study
• Chapter 3. Tools in support of Asset-Based Development
• Chapter 4. Introduction to the IBM Rational Unified Process
• Chapter 5. Asset-Based Development and Asset Governance Rational Method Composer plug-ins
• Chapter 6. Value, impact and return on investment

• Chapter 7. Adopting Asset-Based Development
• Chapter 8. Configure your asset repository
• Chapter 9. Produce reusable assets
• Chapter 10. Consume reusable assets
• Chapter 11. Manage reusable assets

• Chapter 12. Introduction to service assets
• Chapter 13. Producing service assets
• Chapter 14. Consuming service assets
• Chapter 15. Managing service assets

• Chapter 16. Introduction to patterns
• Chapter 17. Produce: model-to-text pattern implementations
• Chapter 18. Produce: model to model
• Chapter 19. Produce: packaging for deployment
• Chapter 20. Consuming pattern implementations
• Chapter 21. Managing pattern implementation assets

• Appendix A. Rational Asset Manager: installation and administration
• Appendix B. Integrating Asset-Based Development into a process
• Appendix C. Additional material

The value and use of Rational Data Architect in SOA

The IBM Information management yesterday announced that the paper "The value and use of Rational Data Architect in SOA" was published today as part of the series "The Information perspective of SOA design"

Here is the abstract from that paper:

Discover how you can use the IBM® Rational® Data Architect, IBM Industry Models and the unified metadata management of IBM Information Server to align process, service, and data models. Use these tools to accelerate your SOA project. The fifth part of "The information perspective of SOA design" series describes the key features of the products that support the data modeling pattern in SOA.

Pattern (reusable) assets classification

The value of applying the canonical modeling pattern in SOA

Today the IBM Information management people published another articles in a series called The Information perspective of SOA design. This article is titled The value of applying the canonical modeling pattern in SOA. Here is the abstract from that paper:

Discover the approach and value of canonical modeling in SOA design. See how the canonical data models can be aligned in SOA with canonical message models. In this fourth article in the "Information Aspect of SOA Related Design" series, learn about the concept's underlying data and message modeling regardless of the technology and tool choices. A future article in this series describes how various IBM® software products can be used to implement the concepts described here.

If you have any questions or suggestions, feel free to contact the authors of the articles in this series Brian Byrne, John Kling, David McCarty, Guenter Sauter, Harald Smith, and Peter Worcester.

The requester side caching pattern implementation specification

developerWorks, SOA and Web services zone has just published the paper, on the requester side caching pattern implementation specification. Part 1 of this article series provided an overview of the requester side caching (RSC) pattern specification, which can help you make and document design decisions around the cache and policies. In this second installment in the series, examine the requester side caching pattern implementation specification, a bridge between the human readable pattern specification from the Gang of Four and the pattern implementation that can be used in a development environment to automate the application of the pattern. From this implementation specification, you have the freedom to create numerous implementations.

Readers can find the article here: The requester side caching pattern implementation specification

The many buckets problem

As I mentioned in my previous post one of the problems that we run into with mapping context to content is the multi bucket problem. Consider the following. At home I organize all my clothes into buckets.

Each bucket has a different item of clothing in it e.g. one bucket contains all my shoes; another bucket contains all my pants; yet another bucket has all my jackets.

How let's imagine for a moment that these are very sophisticated buckets. These buckets have the ability to classify all of the items contained within as well as matching items together. For example my shoe bucket has the ability to classify all of my shoes by color and size as well as matching all of my left shoes with their corresponding right shoes. To top it off these buckets have a shinny panel on the front that gives me detailed information about any item in the bucket. So with my bucket of shoes I can see from the front panel all the information that i need about a certain shoe, such as when I bought it, how often I have worn it etc.

 This is all fine and dandy, but I run into a problem when for example I am invited out to a work related cocktail party. Since this is my first cocktail party I am unsure about what to wear and would like to be able to say to all my clothes buckets, please give me an ensemble to make me look good for this cocktail party. My clothes buckets I am sure will look back blackly at me, with the shinny digital displays, beeping nervously as I start to frantically root through one bucket after another to try and put together some vestige of an outfit that will make me presentable at the cocktail party.

This simple story serves to illustrate the problems inherent in any repository based information management system. However, there are a couple of lessons to be learned from this simple analogy.

1. No one bucket can be responsible for management relationships between the items in its bucket and the items in another bucket as the would quickly be unmanageable for all but the smallest number of bucket items
2. I could put all my clothes into one bucket and call it a clothes bucket, however by treating all of clothes as just generic clothes items I lose a lot of flexibility I has in the multi bucket scenario.

Enabling asset consumability

I live a charmed life where my wife and I get to drive to work together. The drive is about 40 minutes from where we live to where we both work in Cambridge Boston. Because of all of this quality time we often get to talk about what we are doing. So one day last week I was trying to explain to her some of my ideas around mapping of 'context to content'. My wife does not suffer my nonsense easily and told me that this statement made no sense to her. In an effort to explain I gave her a driving analogy since we were already is an automobile.

Behind the wheel of a car I am in a driver context and in this context I need access to driver related information and tooling such as my speed, the condition of my engine, weather conditions, a GPS device, road conditions, including this particular road's speed limit (i.e. the car's context) etc.

This is an example of 'context to content'. I am in a driver context and I need access to the driver relevant and related content.

As a Software developer or a consultant on an engagement I am again in a certain context. The context is now given by the scope of the project and the functional and non-functional requirements for that particular project. This context can also be mapped to content to better help me do my job. For example on insurance project there may be a functional requirement around creating a claims system. Here the functional requirement can be mapped to reusable software assets such as an insurance UML model of a claim system. A nonfunctional requirement on the other hand such as a transactional claims system can maps to another type of reusable software asset such as a software pattern assets to help me made consistent architectural decisions.

The question now becomes how we do automate this context to content mapping for developing software in a consistency manner to allow better consumabitity of reusable asset such as models and patterns?

The problem with implementing such a vision is that we quickly run into what I call the two bucket problem. But more about that in my next blog entry .

The Information perspective of SOA design

• data semantics and in particular Business Glossary + Industry Models
• data structure / canonical data model and in particular RDA + Industry Models (+ RSA)
• data quality analysis and in particular Information Analyzer

The requester side caching pattern implementation specification (part 2)

In this two part blog we will examine the requester side caching pattern implementation specification. This implementation specification is a bridge between the human readable pattern specification (e.g. pattern specifications found in the book on Design pattern from the Gang of Four) and a pattern implementation that can be used in a development environment to automatic the application of the pattern. From this implementation specification we have the freedom to create numerous implementation.

In this second part we will also go into more detail on the Java code that a pattern implementation would need to generate if and when these parameters are bound.

UML to Java Transformation

The transformation implemented for this pattern will extend or reuse the UML to Java transformation.

Service

The AcceleratedService UML class created during pattern instantiation expands to a Java class that implements the Service interface. The constructor of this class will take as argument a service of type Service as argument. The class will also have a private member, service of type Service that is instantiated by the constructor to the 'slow' original service implementation.

Another member of the AcceleratedService class is the cache of type Map. The operations generated in AcceleratedService are operations defined in IService. In addition, we will have a method to retrieve the cache, getCache(), that returns the cache implementation object.

getItem

The getItem pattern parameter expands into an operation in the AcceleratedService class that mirrors its name and signature. This operation takes a 'key' as input and returns an Item as output. The implementation of this method is fully specified during the UML to Java transformation phase. In the fragment below, the method body contains the following variable values:

Item: the return type of the original getItem method.

ItemKey: the argument (that corresponds to the key value) of the original getItem method in the service.

public Item getItem(ItemKey itemKey) {
Item item = (Item)this.cache.get(itemKey);
if (item == null)
item = this.service.getItem(itemKey);
return item;
}

getItems

The getItems pattern parameter expands into an operation in the AcceleratedService class that mirrors its name and signature. The signature of this operation is arbitrary and this can contain any number of parameters and parameter types. The return type of this operation must be a List JDK collection type. The implementation of this method is fully specified during the UML to Java transformation phase. In the fragment below, the method body contains the following variable values:

List: the return type class.

arguments: the argument list of the original getItems method in the service.

ItemKey: the key class/type of the Item class. This value is identified in the argument list of the getItem() pattern parameter.

Item: the item class/type. This value is identified as the return type of the getItem() pattern parameter.

.
public List getItems(arguments) {
List keys = this.getItemKeys(arguments);
Iterator iterator = keys.iterator();
List list = new ArrayList();
while (iterator.hasNext()) {
ItemKey key = (ItemKey) iterator.next();
Item item = this.getItem(key);
list.add(item);
}
return list;
}

getItemKeys

The getItemKeys pattern parameter expands into an operation in the AcceleratedService class that mirrors its name and signature. The signature of this operation is arbitrary and this can contain any number of parameters and parameter types. The return type of this operation must be a List JDK collection type. The implementation of this method is fully specified during the UML to Java transformation phase. In the fragment below, the method body contains the following variable values:

List: the return type class.

arguments: the argument list of the original getItems method in the service.

public List getItems(arguments) {
List keys = this.service.getItemKeys(arguments);
return keys;
}

Users are expected to properly select Qualifiers of getItemKeys, it is recommended that 'Ordered' Qualifier is selected while others de-selecte d, in order to generate a method that expresses multiplicity of return parameter in term of 'java.util.List'.

changeItemKey Parameter

The changeItemKey pattern parameter expands into a specific argument (corresponding the key) of an operation changeItem in the AcceleratedService class. The changeItem operation mirrors the changeItem method declared in the Service interface. The implementation of this method is fully specified during the UML to Java transformation phase. The method fragment below contains the following variables:

Arguments1: zero or more arguments of the original changeItem method that precede the changeItemKey argument.

changeItemKey: the argument that corresponds to the changeItemKey parameter.

Arguments2: zero or more arguments of the original changeItem method that follow the changeItemKey argument.

public changeItem(Arguments1, changeItemKey, Arguments2) {
this.service.changeItem(Arguments1, changeItemKey, Arguments2);
this.cache.remove(changeItemKey);
}

Clustering Parameter

As explained earlier, the clustering parameter effects the kind of cache implementation used by the AcceleratedService. Based on the value of clustering, the implementation of Map will be either an inmemory cache, i.e., class Cache if clustering is false, or class DistributedCache if clustering is true. If clustering is true, the getCache() of the AcceleratedService will look like:

public Map getCache() {
com.ibm.websphere.cache.DistributedMap map = null;
try {
javax.naming.InitialContext ic = new
javax.naming.InitialContext();
Object obj = ic.lookup("services/cache/EmployeeCache");
if (obj != null) {
map = (com.ibm.websphere.cache.DistributedMap)
javax.rmi.PortableRemoteObject
.narrow(obj,
com.ibm.websphere.cache.DistributedMap.class);
System.out.println("successfully retrieved a map");
}
} catch (Exception e) {
System.out.println("failed to retrieve a map");
e.printStackTrace();
}
return map;
}

If clustering is false, a class, Cache that implements the Map interface is generated with an prepackaged code template. The getCache() of the AcceleratedService will look like:

public Map getCache() {
Map map = new Cache(size, timeout);
return map;
}

Cache size Parameter

If clustering is true, the cacheSize parameter expands into the cacheSize value in the properties file used by the underlying WebSphere caching implementation (cacheinstances.properties). If clustering is false, the cacheSize parameter expands into the workingSetSize value in the Custom cache implementation, InMemoryCache.

timeOut Parameter

If clustering is true, the cacheSize parameter expands into the cacheSize value in the properties file used by the underlying WebSphere caching implementation (cacheinstances.properties).

Implementing the specification

Armed with the pattern implementation specification a pattern implementation can now be authored using the RSA pattern engine. The output of this authoring process is an eclipse plug-in that can be used in RSA to automate the application of the requester side caching (RSC) pattern to a model.

The approach to using this pattern follows the model-driven development approach of instantiating the pattern parameters to specific UML model elements of the service or interface. Once the pattern parameters are bound, additional UML elements such as the cache and the cache-aware service proxy are automatically created. A UML-to-Java transformation, invoked on the resulting model, will generate the resulting Java implementation artifacts. This implementation also allows the user to chose between a custom (in memory) user-defined cache or the WebSphere Platform dynamic cache. If the user chooses the latter, the pattern transformation will automatically generate configuration files to be used with dynacache. To see this pattern in action in RSA the reader is referred to the follow article that details how to use the RSA requester side caching (RSC) pattern implementation.

The read can also find a detailed study of the life cycle requester-side caching (RSC) pattern.

The requester side caching pattern implementation specification (part 1)

In this two part blog we will examine the requester side caching pattern implementation specification. This implementation specification is a bridge between the human readable pattern specification (e.g. pattern specifications found in the book on Design pattern from the Gang of Four) and a pattern implementation that can be used in a development environment to automatic the application of the pattern. From this implementation specification we have the freedom to create numerous implementation.

In this first part we will focus of the pattern parameters that need to be identified and the use cases for binding these pattern parameters. In part two we will go into more detail on the lava code that a pattern implementation would need to generate if and when these parameters are bound.

Introduction

In a previous article we have detailed the pattern specification of the requester side caching pattern. A number of implementations of the requester side caching (RSC) pattern can now be created based upon the pattern specification. This pattern could be implemented by hand coding but typically any implementation should provide some level of automation for applying the pattern. The follow three examples showcase different ways that this pattern specification can be implemented:

• A manual implementation (hand coding) that uniquely tailors the design and the implementation for each situation.
• An executable program that automates the application of the pattern in the context of a larger model-driven based development project (e.g., in the context of IBM's Rational Software Architect).
• See Building SOA applications with reusable assets: The requester side caching pattern
• An aspect-based implementation: aspect-oriented programming can be used to implement the caching as a cross cutting concern and can be a simple and useful mechanism for developing applications. This approach would work if aspect-oriented expertise is already available in the organization.
• See Modeling with Aspects: An introduction to modeling with aspects

However, before authoring a pattern, a pattern implementation specification must first be created.

Background

The requester side caching pattern is one of mediating the interaction between one or more clients and one or more data providers. The mediation consists of holding data items that have been produced by the provider(s) and using them to support requests from the client(s). The purpose of the mediation is to speed up and/or reduce the cost of access to the data. This is a very general pattern that has many variations to meet different design goals and issues. The pattern should make it easy for a developer to make design decisions and also to provide a basis for documentation about design decisions made around the cache and policies.

There are a number of issues that must be addressed in any pattern variation. They are:

• Style of access (visible to the client; hidden from the client)
• Item identification (client provided domain key; explicit, well defined key space; computed from data items etc.)
• Populating the cache with data items (on cache misses; on cache misses but with some anticipatory population; fully pre-populated)
• Cache capacity and capacity management (unlimited with respect to the set of data items; easily exceeds to working set of the clients; marginal with respect to the working set of clients)
• Tolerance for data staleness (must be exact; can be slightly out of date; accuracy of the data is a performance concern, not a logical concern)
• Management of data volatility (unchanging data; rarely changing data; rapidly changing data)
• Topology of deployment (single instance; multiple coordinated instances; multiple independent instances)

The requester side caching pattern will facilitate accelerating service operations via caching. The cache used can be an in-memory, custom cache provided by the pattern itself or the WebSphere runtime's caching mechanism (i.e., Dynacache). The runtime cache can be in memory or distributed. One of the goals of this pattern is to make it easy for the developer to programmatically use our runtime caching mechanisms.

This pattern currently assumes that the service operations exposed to the requester include methods for

• retrieving objects from the service provider based on a selection criteria, getItems(criteria)
• retrieving the keys that uniquely identify the objects based on selection criteria, getItemKeys(criteria)
• retrieving a single object based on a key value, getItem(key)
• making changes to objects based on key value, changeItem(..., key, ...)

If operations such as getItemKeys(criteria) and changeItem(key, ...) are not available, one proposal is to have some kind of a facade that will sit between this caching pattern and the service interface where these operations will be inserted.

References

WebSphere Dynamic Cache: Improving J2EE application performance, IBM Systems Journal, Vol 43, No 2, 2004.

Requirements

• The implementation will generate UML model elements as a result of applying the pattern.
• The generated source code should be error free and capable of being executed. Exceptions may be thrown if code marked up with a mandatory 'TODO' has not been updated accordingly by the user.
• This pattern must allow for at least two choices in the caching implementation.
• The pattern should also allow for cache configurations that include cache population, management of data volatility, cache capacity management and item identification.
• The pattern should make it easy for a developer to make design decisions and also to provide a basis for documentation about design decisions made around the cache and policies. The transformation is also constrained by the caching implementations provided by the WebSphere platform, but we will also provide custom caching skeleton code.

Design Constraints

The design and implementation of this pattern will be constrained by the pattern and transformation authoring functions of Rational Software Architect 6.x and 7.x.

Design Artifacts

Pattern Name

Requester side cache

Overview Diagram

The pattern UI will represent this pattern with the following overview diagram.

Group

The pattern will be defined in the following group: SOA Patterns

Pattern Type

An instance of this pattern is of the UML2 element type Collaboration.

Short Description

The Requester side caching pattern provides an accelerated implementation of operations that retrieve information from a service provider. The pattern provides options for cache implementations and caching policies.

Search Keywords

The following keywords will be defined for the pattern to assist the client with locating the desired pattern:

1. SOA Patterns
2. Caching
3. Service requester

Pattern Parameters

Service

• Short Description: The interface/class that contains the operation that we want to accelerate
• Type: Interface/Class
• Multiplicity: 1
• Parameter Dependency: none
• Default Value: none
• Keyword: none

getItem

• Short Description: The operation on the service interface/class that is used to get a single item given an item key.
• Type: Operation
• Multiplicity: 1
• Parameter Dependency: this parameter depends on Service
• Default Value: none
• Keyword: none

getItemKeys

• Short Description: The operation on the service interface/class that is used to get keys given a set of criteria. Note that this operation is essential to be able to implement the accelerated version of the service.
• Type: Operation
• Multiplicity: 1
• Parameter Dependency: this parameter depends on Service
• Default Value: none
• Keyword: none

getItems

• Short Description: Short Description: The operation on the service interface/class that is used to get items given a set of criteria.
• Type: Operation
• Multiplicity: 1
• Parameter Dependency:depends on Service, getItemKeys and getItem
• Default Value: none
• Keyword: none

changeItemKey

• Short Description: The parameter in the change item operation that corresponds to the key of the item.
• Type: Parameter
• Multiplicity: 0..* (if change item is specified then this parameter is required)
• Parameter Dependency: none
• Default Value: none
• Keyword: none

Cache Size

• Short Description: The size of the cache. The size of the cache can be specified as an integer or the character * to denote a unlimited cache.
• Type: LiteralInteger
• Multiplicity: 0..1
• Parameter Dependency: none
• Default Value: 500
• Keyword: none

Clustering

• Short Description: Boolean value: true implies the underlying topology is clustered, false implies the underlying topology is not clustered.
• Type: LiteralBoolean
• Multiplicity: 1
• Parameter Dependency: none
• Default Value: true
• Keyword: none

timeOut

• Short Description: The time out value (measured in milli seconds) after which an item has to be evicted from cache. If this parameter is not specified a time out eviction policy is not used.
• Type: LitrealInteger
• Multiplicity: 0..1
• Parameter Dependency: none
• Default Value: -1
• Keyword: none

The derived parameters from the above listed pattern parameters are: ItemKey class that can be derived as the argument of the getItem operation, Item class that can be derived from the return parameter of the getItem operation, changeItem() method that can be inferred from the changeItemKey, i.e., its owning operation.

Use cases

The actions denoted below occur only at the time the argument is supplied to the pattern parameter. After that, the argument may change irrespective of the constraints of the pattern definition.

Pattern Interactions

Service Parameter

Supply an argument to the parameter

• Add the 'Accelerated' keyword to the argument if the keyword does not exist.
• Create a new class in the same package as the argument.
• The name of the generated class is going to be the name of the argument prepended with the string 'Accelerated'.
• The generated class will realize the argument if the parameter is an interface.
• The generated class will provide implementations for all of the arguments operations if the parameter is an interface. If the parameter is a class, the generated class simply inherits the operations.
• The generated class will provide implementations for the constructor (where Service is the name of the argument)
  AcceleratedService ( service : Service )
• The generated class will have a directed private association to the argument called service
• The generated class will have a have a public getter method for the instance member service.
• The generated class will have a directed private association to a Map interface.
• The generated class will have a have a public getter method for the instance member Map.
• If the clustering parameter is set to false, the constructor creates a cache object of type Cache that is an implementation of the Map interface which supports get, put, remove and clear operations.
• If the clustering parameter is set to true, the constructor (AcceleratedService) gets a cache object of type DistributedCache that is an implementation of the Map interface. The operations of this class are again the same as declared in Map.

Supply the default argument to the parameter

• N/A

Remove an argument from the parameter

• Undo what we did before, in particular remove the newly created class and the associated to the IService parameter. However do not remove the Map interface if it is being referenced by any other modeling element

Replace the parameter argument

• Default to the framework semantics of removing the current argument and supplying the new argument.

Pattern reapply

• Default to the framework semantics of removing the current argument and supplying the new argument.

getItem Parameter

• This argument is only referenced during the UML to Java code transformation.

getItems Parameter

• This argument is only referenced during the UML to Java code transformation.

getItemKeys Parameter

• This argument is only referenced during the UML to Java code transformation.

Change Item Key Parameter

• This argument is only referenced during the UML to Java code transformation.

Cache size Parameter

• This argument is only referenced during the UML to Java code transformation.

Clustering Parameter

• This argument is only referenced during the UML to Java code transformation.

timeOut Parameter

• This argument is only referenced during the UML to Java code transformation.

The derived parameters from the above listed pattern parameters are: ItemKey class that can be derived as the argument of the getItem operation, Item class that can be derived from the return parameter of the getItem operation, changeItem() method that can be inferred from the changeItemKey, i.e., its owning operation.

Building SOA applications with reusable assets: Preferred data source pattern

developerWorks has just published the paper, on the Preferred Data Source Pattern implemenation in Rataional Software Architect. I have examined this Preferred Data Source Pattern specification in previous blog enteries but to restate again: the Preferred Data Source pattern is a Service-Oriented Architecture (SOA) pattern that allows a client to retrieve information from a set of information sources, without knowing (at least at a high level) that multiple sources exist. In this dW article we exmaine in detail one particular implementation of this pattern using Rational® Software Architect. Here is the abstract from that article:

This series explores reusable assets such as recipes, software patterns, and models. The series shows how you can accelerate the development of SOA solutions. This fifth installment in the series explores the preferred data source pattern, which addresses consistency non-functional requirements when implementing reusable services. The preferred data source pattern is a microflow pattern for service aggregation. It was harvested from a real SOA engagement, and it has been reused in several other SOA applications and engagements. This article also demonstrates how you can use a Rational® Software Architect implementation of this pattern in a model-driven development environment to create a new service implementation.

You can find the paper here: Building SOA applications with reusable assets: Preferred data source pattern

Problems in provisioning Enterprise Architectures

In today IT landscape architects are faced with four major problems that need to addressed: 

1. The ever increasing complexity of the Enterprise Architecture
2. The proliferation of vendor, tools and platforms in the Market place.
3. The consumable (or more importantly lack their of) of tools
4. Lack of consistency in the architectural: decision being made by skilled developers.

Let us first address the increasing complexity of Enterprise Architectures. Over the last 15 years businesses have had to deal with an ever increasing complexity of the IT offering. In the 1980s starting with the simple stand alone application and moving on to the a client server environment. In the early 1990s CORBA brought us into a distributed environment. However, developer often spent most of their time writing the low level plumbing and often neglected all important business logic. 

So in the mid 90’s Java came along with J2EE and EJBs and gave us the managed container to deal with all that underlying pluming. With the plumbing out of the way it allowed architects to focus more on the structure of the business logic and the notion of an-distributed architecture for the masses became a reality.

It is also interesting to note that the 90s also gave rise to a number of successful architectural styles or pattern such as the model-view-controller, n-tier architecture, requester-response and publish-subscribe

Finally in this century we have a new architectural style or pattern called SOA. This new architectural style build on top of the previously successful architectural style but focuses more on the notion of a reusable service rather that the transactional focused architectures of it predecessor. SOA is in many ways a reaction to the dot com bubble and it grounds architects firmly in the problems of the business domain: A good way to think about SOA is as follows. "Alignment of business and IT, to achieve a flexible business model, to enable the business be more agile, in an aggressively changing market?"

In my next blog we will look at the problems of proliferation of vendor, tools and platforms in the market place.

Inside the preferred data source pattern

developerWorks has just published the paper, on the Preferred Data Source Pattern, a Service-Oriented Architecture (SOA) pattern that allows a client to retrieve information from a set of information sources, without knowing (at least at a high level) that multiple sources exist.

The Preferred Data Source Pattern, or Preferred Source Pattern, is a microflow pattern for service aggregation. The pattern allows a client to retrieve information from a group of information sources without the need to understand, at least at a high level, that multiple sources exist.

Consider the following situations where multiple data sources must appear as one:

• A company has multiple sources of information, some of which are more expensive to access than others (for example, a local parts database and a remote parts database).
• A company upgrades its IT systems and, in doing so, introduces new sources of information that it must use in conjunction with old sources (for example, customers).
• One or more similar businesses merge, and all have somewhat dissimilar data representing the same entities, such as customers.
• Any individual entity has some enterprise-unique identifier that's part of the record (for example, a customer number or SKU).

You can find the paper here: http://www-128.ibm.com/developerworks/webservices/library/ws-patterns/index.html

The preferred data source pattern

The preferred data source pattern, provides the ability for a client to retrieve information from a set of information sources, without the need to understand that there are multiple underlying sources. One of the sources is identified as the preferred source, and the others are considered alternate sources, used only when the preferred source cannot provide the desired information. This pattern specification will be published in full on dW (May 2007) but here is a preview of it until then.

Context

Consider the following situations where multiple sources of data must be made to appear as one.

• a company has multiple sources of information, some of which are 'more expensive' to access, e.g., a local and remote parts database

• a company upgrades its IT systems, and in doing so, introduces new sources of information that must be used in conjunction with old sources, e.g., customers

• one or more similar businesses merge, all have somewhat dissimilar data representing the same entities, e.g., customers

• any individual entity is assumed to have some "enterprise unique" identifier that is part of the record, for example, a customer number or SKU

There is also assumption that the integration of the above scenarios in a done in the context information management SOA web services environment.

Problem

How can a client retrieve information from a set of disparate information sources, without the need to understand (at least at a high level) that there are multiple sources?

Solution

The preferred data source pattern provides the ability for a client to retrieve information from a set of information sources, without the need to understand (at least at a high level) that there are multiple sources. One of the sources is identified as the preferred source, and the others are considered alternate sources, used only when the preferred source cannot provide the desired information. Figure 0 shows the the relationship between the facade and the adapters

The information obtained from any source is assumed to be in the form of "records" that describe entities such as customers or parts. Further, any individual entity is assumed to have some "enterprise unique" identifier that is part of the record, for example, a customer number or SKU.

The heart of the solution is a facade; the client interacts only with the facade, which hides the fact that there are multiple data sources. The facade interface matches that of the preferred source (more on that below). The preferred interface contains one or more operations that allow the client to find (read) information matching various criteria. A find operation returns 0..n records that match the criteria. It is important to understand that no matter which source provides the information, it is possible that none of the returned records are the desired record. Consider a scenario where a store clerk searches in a nation-wide company database for customers with the name "John Smith;" the find operation could return 20 John Smiths, but none are the John Smith in front of the clerk. The client must depend on additional interactions with the users to determine whether any of the returned records are the desired record.

The preferred source pattern assumes that an information source has a one or more 'find' operations that return zero or more instances of the entity record, or perhaps a subset of the entity record. The information source may have one or more 'write' operations that allow a client to create and update entity records.

Find operations

Figure 1 shows a sequence diagram for a find operation in the pattern. The client invokes the facade. The facade invokes the preferred information source. If there are no matches from that source, the facade invokes the alternate information sources in a pre-defined order until matches are found, or until all the alternative sources are exhausted. Once a match is found, or all sources are exhausted, the facade returns to the client. Note that for clarity, I've not shown the synchronous returns.

In its simplest form, the preferred source, and thus the pattern, supports only find (read) operations. A virtual catalog capability might leverage such a 'read-only' pattern, as there is no need (or perhaps no ability) to update the preferred source. The description for the simplest form must include:

The WSDL document describing the preferred source and all alternate sources. The interface (port type) of the preferred source is used by the facade and all alternate sources. If an alternate source does not natively expose the same interface, a transform pattern is applied to the source, but the transform is out of scope. The WSDL for the alternate sources must differ from the preferred source at least in the endpoint address; it may differ in the binding(s) as well, with a bit more work.
The schema describing the entity record and any other parameters used in the interface. Note that the schema will be defined by or imported by the WSDL document. As indicated above, it is assumed that an entity record includes a unique ID.
Identification of the 'find' operations to which the pattern will be applied. All other operations will be treated as pass-through.
A list showing the order in which the alternate sources are invoked. It is of course possible to have a single list of WSDL documents for the services and the first in the list is assumed to be the preferred source.

The value of patterns: A case study on the value and lifecycle of patterns