Dr. Eoin Lane, Senior Solution Engineer, is the lead for harvesting and developing of application pattern from key IBM SOA engagements and driving those patterns through IBM pattern governance process to accelerate adoption. Eoin also specializes in Model Driven Development (MDD), asset based development and Reusable Asset Specification (RAS) to facilitate SOA development.

Requiem for a developer

The more I think about SOA the more I believe there is only one real logical conclusion and that is less jobs for us IT folks. Now before I descend into the doom and gloom as to why we are all unwittingly developing ourselves out of a job here I want to go back to school first. One of the poems I studied in high school (we called it secondary school where I came from) was T.S. Eliot - Mcavity: The Mystery Cat

You'll be sure to find him resting, or a-licking of his thumbs,
Or engaged in doing complicated long division sums.

I always loved the idea of an alley cat capable of doing complicated long division sums. We will come back to Mccavity again later but for now I want to share an experience I had recently while helping a customer with SOA.

I have been involved with helping an insurance company migrate to a business driven SOA. Late last year I helped their personal lines business identify and specify their reusable services. More recently I was back doing some service modeling for their commercial lines.

This gave me a better view of their overall service portfolio and allowed the customer to see the value of standardizing on a common message model and also to see which services are being reused the most. One of the services we identified was a customer relationship management (CRM) kind of service. As the name suggests this service was typically used for recording customer information and preferences. This is a good example of a service is reused by many business process and services and is also reused across the two silos of the personal and commercial lines.

It is not usual or uncommon for a company like this to have one or more CRM like applications in each of the information silos i.e. personal lines would have and CRM application, commercial lines would have another one, etc. Depending on the organization this application could be home grown i.e. written and maintained by the company own IT department, or there could be some third party application that is maintained by the IT department.

As the company now begins to agree and standardize on a common service definition for the CRM application, the question becomes, who will implement this interface. There are at least a couple of answers to this question.

1. The first is that there will be a common interface but the existing backend application in each of the silos will be responsible for provisioning this service and it is  up to the message infrastructure to do the necessary ‘content based’ routing to make sure the correct application is invoked.
   
2. The second answer however is where things get a bit hairy. Now let us consider this from the company CIO perspective, he very proud of all the work that went into agreeing on and specifying a CRM service interface that is now used by everywhere across the company. However, he still has a number of CRM applications, being used to provision this service, that need to maintained, upgraded, debugged etc. What will really hit him, at this stage, is the fact that this is an insurance company and yes of course their customers and their relationship with their customers are their top priority but managing and maintaining this information is not, nor will every be core to their business. Indeed there are a lot of other companies out there whose core business is managing and maintaining customer information. Once he realizes this, it is only a matter of time before he decides to sun set all their siloed CRM application that have been consuming so much of his time and recourses and either bring in one third party CRM application or even more realistically out source the implementation of their CRM service to a third party like Saleforce.com.

It does not take a rocket scientist to do the math here.  If we start to consider all of the CRM application in all the silos of this insurance company, of every insurance company, indeed of ever company which does not have managing and maintain customer relationship information at it core business (because every other CIO will come to the same conclusion) now being out sourced to a third party. That equates to a lot of IT staff with a lot more time on their hands. Of course I have just focused on CRM here; the same could be said about claim management. I do not want to talk for the insurance industry here but it is not unreasonable to assume that underwriting is the core competency of insurance and that in theory once the service specifications had been specified then all of the non underwriting services could be out sourced.

The second option taken to its extreme scenario could even lead to the CIO putting himself out of a job and where insurance companies of future would be staffed by small number of very smart number cruncher, who just like my old pal Mccavity are capable of doing some very complication long division sums.

More from Services-based enterprise integration patterns made easy

Create collaborative and dynamic method content using Web 2.0

developerWorks, SOA and Web services zone has just published the first in a series of papers that looks at addressing some of the ideas around context to content to enable asset consumability that I have been talking about on this blog, This paper titled: Create collaborative and dynamic method content using Web 2.0, talks about how to leverage Web 2.0 technologies to extend software development process content, which is typically published static as HTML. This article describes how you can develop the ability to collaboratively edit method content and have access to the latest dynamic content within a method context.

For this paper, and indeed many other papers, we have been fortune enough to work with developerWorks own, Patrick Flanders, and Ashleigh Brothers who do such a professional job in terms of reviewing, editing and laying out the content.

Among the school children

A couple of weeks ago I was giving a talk on enabling (pattern) asset consumability at an IBM hosted best practices conference down in Palisades, New York. The talk was scheduled to be at the end of the day so I knew my audience brains would be fried and overload by technical content by this time. Also, the title of my talk enabling (pattern) asset consumability would make the best of us yawn. So I decided on a different tack here, I am still not sure how successful it was, since only a handful of poor souls made it to my talk, but here it is anyway.

Recently I have read two excellent book on speech giving. The first is Give Your Speech, Change the World and the second on is Make to Stick. Now where there first book is excellent in detailing how to actually give presentation, the second book excels in detailing how to make your messages, in those presentations, stick to your audience. So for my talk I decided I would shake things up and open with a poem in a effort to exercise the right side of my their brains. The poem titled "Among the school children" from one my favorite poets W. B Yeats. Now this poem is long over 8 stanzas, so I just focused on one of the stanzas, here it is:

Plato thought nature but a spume that plays
Upon a ghostly paradigm of things;
Solider Aristotle played the taws
Upon the bottom of a king of kings;
World-famous golden-thighed Pythagoras
Fingered upon a fiddle-stick or strings
What a star sang and careless Muses heard:
Old clothes upon old sticks to scare a bird.

Here Yeats is examining the philosophies of some of the worlds famous philosophers from Plato to Pythagoras, yet with all their knowledge and insights, they all still withered and died away. In his later poetry, Yeats uses this recurring theme of a scarecrow to symbolize his own mortality. Here from the same poem:

Better to smile on all that smile, and show
There is a comfortable kind of old scarecrow.

As I said I am not sure how successful this technique was, my audience of five just was enough to swing it either way, but I would certainly be interested to know of other unconventional speech giving techniques that you have used in the past that have worked?

We will be publishing a paper on this talk soon and will post the link as soon as it becomes available.

A common message

1. The first best practice is simply to have one. The business and the IT need to agree upon an enterprise message format, or lingua franca across the organization. Thus what was traditionally a silo-ed organization in terms of personal and commercial insurance (each has a difference via of what a customer/party is) could now start to share service based on a common understanding about what what was a claim, a policy or indeed a customer/party. This really is not news to anyone starting to mature in the SOA space, however, the next best practice is a bit trickier.
2. The second best practice is not to build this enterprise common message model yourself. This can not be overstated. Building a enterprise strength common message model in a big job. To get a sense as to how big, consider IBM Insurance Application Architecture or IAA models. Now I am not here to plug these models, but they have been built up over ten years worth of experience, of practitioners, in the insurance industry. As a result these models are highly normalized and contain some of the best patterns and lessons learned and are what I would consider a mature enterprise model. There are many other enterprise strength message models out there such as ACORD, SID (telco), STAR (automotive) . One also needs to differentiate between industry standard models and priority models and here again I want to outline a best practices in terms of their usage. Industry standard model such as ACORD have been debated and agreed upon by concerned parties within an industry vertical, these models are there typically to level the playing field and provide a good, typically flat, model structure to enable B2B exchanges between companies in that particular business vertical. Priority models on the other hand such as IAA and IBM Information FrameWork Service Models (IFW), are highly normalized models, and are there to provide a company with core differentiation within the marketplace. The best practices then become to use a priority model internally within the organization, to define the internal services and as a lingua franca for the messaging infrastructure and then to use one or more industry standard models to define services that are exposed external to the organization and typically involved in B2B exchanges and delegate to the message infrastructure to convert between them.
3. Finally the enterprise message model must be managed and governed. New versions of these message model are being released all the time as new markets and opportunities arise and your company needs a coherent and consistent way of managing this change. Let us go back to our UN example and imaging that the Irish language was continuously changing (it is not, the Irish language in a considered a dead language), think of the nightmare for the interpreters having to continuously learn a new version of the common language.

For those you foolish enough to want to learn Irish, to champion the cause of having the UN adopt it as a common international language, if refer you to Des Bishop web site

Slán go fóill...

SOA information as a service pattern specifications

Guenter Sauter et al have three excellent SOA information as a service pattern specifications papers on data federation, data consolidation, and data cleansing. You can find these three papers on developerWorks:

• Data federation pattern: The data federation pattern virtualizes data from multiple disparate information sources. The pattern creates an integrated view into distributed information without creating data redundancy while federating both structured and unstructured information. This article describes the federation of structured information (data) with a focus on the SOA context. This pattern specification helps data and application architects make informed decisions on data architecture and document decision guidelines.
• Data consolidation pattern: The data consolidation pattern specification helps data and application architects make informed architectural decisions and improve decision guidelines. In this article you will see how you can apply the pattern in the SOA context. The primary business driver for the data consolidation pattern, also referred to as the data population pattern, is to gather and reconcile data from multiple data sources before this information is needed. To do so, it extracts data from one or more sources, transforms that data into the desired target format, and loads it into some persistent data target. The prepopulation of a persistent target is a key differentiator between this pattern and the data federation pattern.
• Data cleansing pattern: The data cleansing pattern specifies rules for data standardization, matching, and survivorship that enforce consistency and quality on the data it is applied against. The data cleansing pattern validates the data (whether persistent or transient) against the cleansing rules and then applies changes to enforce consistency and quality. In this article, you learn to apply the data cleansing pattern within a Service-Oriented Architecture (SOA) context. This pattern specification helps you, as a data or application architect, to make informed architectural decisions and to improve decision guidelines.

Services-based enterprise integration patterns made easy

A new artilce from Dr. Waseem Roshen, in the series titled: Services-based enterprise integration patterns made easy, appeared today on dW SOA and Web services zone. Here is the abstract:

This series of articles explains services-based enterprise integration patterns in an easy-to-understand, step-by-step way. In this installment, Part 1 of the series, you learn about the two earliest integration patterns—data sharing only and remote procedure call (RPC)—which help introduce the concepts of service provider and service consumer, platform independence, and connectivity. Exploring RPC helps you get familiar with the basic steps necessary for two applications to share functionality. This article also includes a general description of the concepts of loose coupling, code reuse, and layering and componentization. Part 2 of the series will continue the discussion of the early patterns, while Parts 3 and 4 cover the Service-Oriented Architecture (SOA)-based integration patterns, including examples.

The asset graveyard

One of my favorite movies of all time is Sergio Leone spaghetti western classic: The Good, The Bad and the Ugly. Part of the appeal of this movie is the simplicity of the plot; at its heart it is, a treasure hunt, with the three main characters, the good, the bad and the ugly hunting for confederate gold. Between them they know the location of the treasure, buried in a grave in a cemetery, yet only their combined knowledge will lead them to the prize. Towards the end of the film the Ugly a character called Tuco, played by Eli Wallach, comes upon the graveyard, called Sad Hill, and here in one of cinema truly great moments, backed by a unforgettable score, he begins his frantic and haphazard search for the grave of Arch Stanton.

What is impressive about this scene is the sheer scale and size of the graveyard itself, a conservative estimate would put it at over 2000 graves, arranged in a circular fashion. Tuco starts from the center of Sad Hill, where the graves are oldest, and works his way outward initially in concentric circles but soon his search becoming more haphazard as he despairs at the magnitude of the task. Finally, after a dizzy and Groundhog Day like scene of multiple graves passing by he come upon the grave of Arch Stanton, only to find it devoid of any gold, indeed all that remains is a rotting corpse.

Today, this story is echoed in ever IT projects I have ever been a part of, one of my favorites is from within my own group in IBM. It should be noted here that our team is geographically dispersed so although we talk on the phone regularly,we do not get the chance to chew the fat at lunch or around a water cooler. A good friend and colleague of mine on our team, lets call him Bob, was leading the engineering of one the first SOA service registry for an automotive customer in California. One requirement the customer had for the service registry that it performs within certain limits. Unknown to Bob, our group has tackled a very similar problem a couple of years earlier on another engagement. We realized the importance of this work and created a number of assets around it. These assets feel under the broad classification of the requester side caching pattern of which we created pattern specifications, pattern implementation, pattern documentation (including, dW articles, flash movies and a detailed case study). A full listing of all of these and other pattern asset related articles can be found on my home page.

Bob unaware of the previous asset work we had done in this space and having I am sure ran in circles around his own repository graveyard a number of times, he did what any other good architect would do, he started from scratch and reengineered the exact same asset as we had two years earlier.

Just like Tuco desperately searching Sad Hill for the confederate gold, Bob too faced the same dilemma, in fact I have witness that same dilemma on every IT engagement and software project I are involved in. I dare say that this dilemma is not evenexclusive to IT but that you will find the exact same problem is almost every human endeavor whether it be an engineer building a rocket to put a man on mars to a lawyer drafting claims for a patent application, How do we provide the right assets,the content, to help solve the problem at hand, the context.In other words, how do we automate a context to content mapping to suggest the best assets to our architects, engineers, lawyers to help them?

IBM Redbook on Strategic Reuse with Asset-Based Development

A good friend of mine Lee Ackerman let me know last week that a IBM have just publicly released a draft of a new Redbook, titled: "Strategic Reuse with Asset-Based Development". The book is available from: http://www.redbooks.ibm.com/redpieces/abstracts/sg247529.html?Open

The Redbook is structured as follows:

• Chapter 1. Introduction to Asset-Based Development
• Chapter 2. ZYX Electronics case study
• Chapter 3. Tools in support of Asset-Based Development
• Chapter 4. Introduction to the IBM Rational Unified Process
• Chapter 5. Asset-Based Development and Asset Governance Rational Method Composer plug-ins
• Chapter 6. Value, impact and return on investment

• Chapter 7. Adopting Asset-Based Development
• Chapter 8. Configure your asset repository
• Chapter 9. Produce reusable assets
• Chapter 10. Consume reusable assets
• Chapter 11. Manage reusable assets

• Chapter 12. Introduction to service assets
• Chapter 13. Producing service assets
• Chapter 14. Consuming service assets
• Chapter 15. Managing service assets

• Chapter 16. Introduction to patterns
• Chapter 17. Produce: model-to-text pattern implementations
• Chapter 18. Produce: model to model
• Chapter 19. Produce: packaging for deployment
• Chapter 20. Consuming pattern implementations
• Chapter 21. Managing pattern implementation assets

• Appendix A. Rational Asset Manager: installation and administration
• Appendix B. Integrating Asset-Based Development into a process
• Appendix C. Additional material

The value and use of Rational Data Architect in SOA

The IBM Information management yesterday announced that the paper "The value and use of Rational Data Architect in SOA" was published today as part of the series "The Information perspective of SOA design"

Here is the abstract from that paper:

Discover how you can use the IBM® Rational® Data Architect, IBM Industry Models and the unified metadata management of IBM Information Server to align process, service, and data models. Use these tools to accelerate your SOA project. The fifth part of "The information perspective of SOA design" series describes the key features of the products that support the data modeling pattern in SOA.

Pattern (reusable) assets classification

The value of applying the canonical modeling pattern in SOA

Today the IBM Information management people published another articles in a series called The Information perspective of SOA design. This article is titled The value of applying the canonical modeling pattern in SOA. Here is the abstract from that paper:

Discover the approach and value of canonical modeling in SOA design. See how the canonical data models can be aligned in SOA with canonical message models. In this fourth article in the "Information Aspect of SOA Related Design" series, learn about the concept's underlying data and message modeling regardless of the technology and tool choices. A future article in this series describes how various IBM® software products can be used to implement the concepts described here.

If you have any questions or suggestions, feel free to contact the authors of the articles in this series Brian Byrne, John Kling, David McCarty, Guenter Sauter, Harald Smith, and Peter Worcester.

The requester side caching pattern implementation specification

developerWorks, SOA and Web services zone has just published the paper, on the requester side caching pattern implementation specification. Part 1 of this article series provided an overview of the requester side caching (RSC) pattern specification, which can help you make and document design decisions around the cache and policies. In this second installment in the series, examine the requester side caching pattern implementation specification, a bridge between the human readable pattern specification from the Gang of Four and the pattern implementation that can be used in a development environment to automate the application of the pattern. From this implementation specification, you have the freedom to create numerous implementations.

Readers can find the article here: The requester side caching pattern implementation specification

The many buckets problem

As I mentioned in my previous post one of the problems that we run into with mapping context to content is the multi bucket problem. Consider the following. At home I organize all my clothes into buckets.

Each bucket has a different item of clothing in it e.g. one bucket contains all my shoes; another bucket contains all my pants; yet another bucket has all my jackets.

How let's imagine for a moment that these are very sophisticated buckets. These buckets have the ability to classify all of the items contained within as well as matching items together. For example my shoe bucket has the ability to classify all of my shoes by color and size as well as matching all of my left shoes with their corresponding right shoes. To top it off these buckets have a shinny panel on the front that gives me detailed information about any item in the bucket. So with my bucket of shoes I can see from the front panel all the information that i need about a certain shoe, such as when I bought it, how often I have worn it etc.

 This is all fine and dandy, but I run into a problem when for example I am invited out to a work related cocktail party. Since this is my first cocktail party I am unsure about what to wear and would like to be able to say to all my clothes buckets, please give me an ensemble to make me look good for this cocktail party. My clothes buckets I am sure will look back blackly at me, with the shinny digital displays, beeping nervously as I start to frantically root through one bucket after another to try and put together some vestige of an outfit that will make me presentable at the cocktail party.

This simple story serves to illustrate the problems inherent in any repository based information management system. However, there are a couple of lessons to be learned from this simple analogy.

1. No one bucket can be responsible for management relationships between the items in its bucket and the items in another bucket as the would quickly be unmanageable for all but the smallest number of bucket items
2. I could put all my clothes into one bucket and call it a clothes bucket, however by treating all of clothes as just generic clothes items I lose a lot of flexibility I has in the multi bucket scenario.

Enabling asset consumability

I live a charmed life where my wife and I get to drive to work together. The drive is about 40 minutes from where we live to where we both work in Cambridge Boston. Because of all of this quality time we often get to talk about what we are doing. So one day last week I was trying to explain to her some of my ideas around mapping of 'context to content'. My wife does not suffer my nonsense easily and told me that this statement made no sense to her. In an effort to explain I gave her a driving analogy since we were already is an automobile.

Behind the wheel of a car I am in a driver context and in this context I need access to driver related information and tooling such as my speed, the condition of my engine, weather conditions, a GPS device, road conditions, including this particular road's speed limit (i.e. the car's context) etc.

This is an example of 'context to content'. I am in a driver context and I need access to the driver relevant and related content.

As a Software developer or a consultant on an engagement I am again in a certain context. The context is now given by the scope of the project and the functional and non-functional requirements for that particular project. This context can also be mapped to content to better help me do my job. For example on insurance project there may be a functional requirement around creating a claims system. Here the functional requirement can be mapped to reusable software assets such as an insurance UML model of a claim system. A nonfunctional requirement on the other hand such as a transactional claims system can maps to another type of reusable software asset such as a software pattern assets to help me made consistent architectural decisions.

The question now becomes how we do automate this context to content mapping for developing software in a consistency manner to allow better consumabitity of reusable asset such as models and patterns?

The problem with implementing such a vision is that we quickly run into what I call the two bucket problem. But more about that in my next blog entry .