When business analysts model business process they tend to capture the sequence of tasks and events without trying to structure into patterns. As an example sequential workflows that usually happen between different actors are mixed with very dynamic interactions such as screen navigations with lots of back & forth or context switch actions. Event driven reactive processing is mixed with service proactive sequence handling. The risk of such mix is to face modeling limitations, and to lead implementation team to the wrong technology selection with an induced higher cost. The following picture describe the essential patterns and the corresponding technology.
To differentiate Service Oriented Architecture and Event Driven Architecture the classical gang of four work on Design Patterns can be used as described by the following picture.
There are many ways to implement polymorphic or variable information and relevant supporting technologies
Here is a list of ways to support Polymorphic Information:
fiammante 100000A8UA Tags:  bpm business_process service variability polymorphism soa 2,901 Views
Variable Services or Service Polymorphism, in the context of object oriented programming, is the ability of one type, A, to appear as and be used like another type, B.
When applied to service invocation, we are talking about the ability to invoke one single service facade F, but actually invoke other services A, B, C etc, without the caller being aware.
This may look simple for Object Oriented Practictionners but Web Services Interoperability standard WS-I states that "Operation name overloading in a wsdl:portType is disallowed by the Profile".
The consequence is that we have to find service variability techniques that use Enterprise Service Bus mediations or other dynamic endpoint resolutions to solve the problem.
This enables the consumer to interact consistently with a generic service, and abstracts the many possible implementations, or specific services, which may ultimately handle the request.
In my book I have a full chapter on service variability and my colleague Scott Glen has written detailed articles in IBM DeveloperWorks address ways to implement service polymorphism.
Seventh of Seven practices for Dynamic Process: Use Information & Event Centric Processes where appropriate
There are many processes that are information or business object centric. In such cases the entities have their own lifecycle and interact with the external world using services. Process analysts must however be careful on ensurign that they are now mixing business ownernship and separate the correctly the facettes that are owned by different owners in an organization. As an example the Telco standard information model clearly differentiates "the Customer Order" owned by customer facing organization from the "Service Order" and "Resource Orders" owned by other organizations in the enterprise.
Then the approach analyzes the state charts for each entity.
This approach can also be taken when different organization have to share a common entity such as a customs manifest declaration. The public entity and its public life cycle can be passed as an SCXML State Chart XML W3C SCXML Standard document. A reference implementation of SCXML available from Apache SCXML project.
The following picture describes such document bei exchanged between different organization and agencies, each with its own IT infrastructure. The only requirement is that each of those organization is able to interpret the SCXML standard to make out how the entity should be handled. It addition the document can carry the history of events that enables each stakeholder to reconstruct a monitoring view.
Managing the complexity of business processes
Enterprises embarking on the business process management journey must ensure that they keep the gains that business processes provide by controlling the cost of their life cycle. Particularly, a process quality approach is essential to enable low-cost changes. This article should interest business architects, and IT managers and architects as it discusses an approach to controlling the development and maintenance efforts for business processes by limiting their complexity.
See the rest of the article on SOA Magazine .
My book got a five star review from http://books.dzone.com/reviews/dynamic-soa-and-bpm-best
InfoQ has just published a review of my book together with a Q&A here is the link of the interview and book exceprt:
InfoQ article link.
I wish you happy holiday with your family and the best for 2010.
Here are the URLs to the references I use in my book.
Wikipedia Enterprise Architecture, http://en.wikipedia.org/wiki/Enterprise_Architecture
The Open Group, http://www.opengroup.org/.
Zachman Framework, http://www.zachmaninternational.com/index.php/the-zachman-framework
Telemanagement Forum eTOM standard,
APQC portal process classification framework, http://www.apqc.org/portal/apqc/site/?path=/research/pcf/index.html .
IBM Component Business Modeling,
Telco Application Map Standard from Telemanagement Forum, http://www.tmforum.org/page33552.aspx .
OGSI Globus Grid Services Toolkit, http://www.globus.org/toolkit/ .
IBM SOA Foundation: An architectural introduction and overview 12,
The RFC 3444, http://tools.ietf.org/html/rfc3444 , titled “On the Difference between Information Models and
Data Models,” states the main purpose of an IM is to model managed objects at a conceptual level, independent
of any specific implementations.
BPMN specification 1.1, http://www.bpmn.org/Documents/BPMN%201-1%20Specification.pdf .
Open SOA Collaboration, http://www.osoa.org
IBM CICS transactional server, http://www-01.ibm.com/software/htp/cics/.
Forum eTOM reference, http://www.tmforum.org/browse.aspx?catID=1647 .
APQC portal process classification framework, http://www.apqc.org/portal/apqc/site/?path=/research/pcf/index.html .
BPMN Private and Abstract processes page 13 and Figure 7.1 and 7.2 of
XPDL presentation, www.xpdl.org/tdocs/200809_KMWorld/200809_SJ04_XPDL_BPMN.ppt .
WebSphere Business Modeler XML schema reference,
U.S. Census bureau statistics, http://www.census.gov/csd/susb/susb06.htm
IBM announcement letters, http://www-01.ibm.com/common/ssi/index.wss .
U.S. Federal Data Reference Model, http://www.whitehouse.gov/omb/assets/egov_docs/DRM_2_0_Final.pdf
IBM Financial Services models, https://www-03.ibm.com/industries/financialservices/doc/content/bin/fss_bdw_gim_0306.pdf .
Telemanagement Forum Information Framework (SID), http://www.tmforum.org/DocumentsInformation/1696/home.html .
RDF Vocabulary Description Language, http://www.w3.org/TR/rdf-schema/ .
Rational Fabric tooling for UML to OWL, http://www.ibm.com/developerworks/rational/downloads/08/rsa_webmodtool/index.html .
URI RFC standard, http://www.rfc-editor.org/rfc/rfc3305.txt .
ISO 20022 Universal financial industry message scheme,
XML Linking Language (XLink), http://www.w3.org/TR/xlink/ .
Example of IBM Sec Filing using XBRL , http://www.sec.gov/Archives/edgar/data/51143/000110465908071167/ibm-20081028.xml
IBM Master Data Management, http://www-01.ibm.com/software/data/ips/products/masterdata/ .
Relationships in WebSphere, http://www.ibm.com/developerworks/websphere/library/techarticles/0605_lainwala/605_lainwala.html
Services Data Object standard, http://www.osoa.org/display/Main/Service+Data+Objects+Home .
Introduction to Service Data Objects, http://www.ibm.com/developerworks/java/library/j-sdo/
Adaptive Business Objects, http://www.research.ibm.com/people/p/prabir/ABO.pdf .
W3C Web Services home, http://www.w3.org/2002/ws/Activity
Erich Gamma et al., Design Patterns: Elements of Reusable Object-Oriented Software, (Addison-Wesley, 1995)
WSDL standard, http://www.w3.org/TR/wsdl
WS-I Basic Profile, http://www.ws-i.org/Profiles/BasicProfile-1.0-2004-04-16.html .
WS-I Attachment profile, http://www.ws-i.org/Profiles/AttachmentsProfile-1.0.html
Open SOA SCA C++ binding, http://www.osoa.org/download/attachments/28/SCA_ClientAndImplementationModel_Cpp_V09.pdf?version=1 .
OMG’s CORBA Interface Definition Language, http://www.omg.org/cgi-bin/doc?formal/02-06-39
Obtaining the WSDL for a PHP SCA component offering, http://www.php.net/manual/en/SCA.examples.obtaining-wsdl.php .
Open SOA SCA with PHP, http://www.osoa.org/display/PHP/SCA+with+PHP
Fielding’s dissertation on Restful services, http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm .
IETF HTTP 1.1 Hypertext Transfer Protocol standard, http://tools.ietf.org/html/rfc2616
ObjectWeb fractal model, http://fractal.objectweb.org/documentation.html .
Definition of rete: synonym of plexus, http://wordnetweb.princeton.edu/perl/webwn?s=rete
Telemanagement Forum MTOSI standard, http://www.tmforum.org/mTOPMTOSIDocuments/2320/home.html .
Open SOA SCA Java Connector Architecture binding, http://www.osoa.org/download/attachments/35/SCA_JCABindings_V1_00.pdf?version=2
WebSphere Dynamic Process Edition, http://www-01.ibm.com/software/integration/wdpe/ .
Zapthink SOA Software Forms an ESB Federation, http://www.zapthink.com/news.html?id=1949
WebSphere DataPower SOAAppliances, http://www-01.ibm.com/software/integration/datapower/ .
WebSphere Business Services Fabric, http://www-01.ibm.com/software/integration/wbsf/index.html
OWL Web Ontology Language, http://www.w3.org/TR/owl-features/ .
ILOG JRules, http://www.ilog.com/products/jrules/
JSR 94 Rules engine API, http://jcp.org/aboutJava/communityprocess/final/jsr094/index.html .
WebSphere Infocenter description of the Work Area Service, http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/topic/com.ibm.websphere.express.doc/info/exp/workarea/concepts/cwa_overview.html .
Building an aggregation function using WebSphere ESB, http://www.ibm.com/developerworks/websphere/library/techarticles/0708_butek/0708_butek.html
IA12: WebSphere Message Brokers for z/OS - CICSRequest node, http://www-.ibm.com/support/docview.wss?rs=171&uid=swg24006950&loc=en_US&cs=utf-8〈=en .
Australian Government Standard Business Reporting
Common Information Model for Energy and Utilities, http://cimug.ucaiug.org/default.aspx .
ACORD Insurance Data Standard, http://www.acord.org/home/
Telemanagement Forum clickable business process framework, http://www.tmforum.org/BusinessProcess-Framework/6775/home.html .
WS-BPEL 2.0 Standard, http://docs.oasis-open.org/wsbpel/2.0/wsbpel-v2.0.html
Open SOA SCA standard, http://www.osoa.org/display/Main/Home.
WS-BusinessActivity standard, http://docs.oasis-open.org/ws-tx/wstx-wsba-1.1-spec-errata-os.pdf
IBM WebSphere Business Service Fabric Modeling Tool, http://www.ibm.com/developerworks/rational/downloads/8/rsa_webmodtool/index.html .
WebSphere Registry and Repository Impact analysis,
Web Services Reliable Messaging, http://docs.oasis-open.org/ws-rx/wsrm/200608 .
Web Services Business Activity (WS-BusinessActivity), http://xml.coverpages.org/wstx-wsba-1.1-rddl.html
WebSphere Service Registry and Repository V6.0 documentation,
WebSphere Business Process Engine API,
ITCAM for SOA,
User Defined XPath Function (UDXF) in WebSphere
Business Monitor, http://www.ibm.com/developerworks/library/i-bam617/
OASIS WSFED technical committee, http://www.oasisopen.org/committees/documents.php?wg_abbrev=wsfed .
Liberty Alliance ID-FF, http://www.projectliberty.org/resource_center/specifications/liberty_alliance_id_ff_1_2_specifications
Understanding SOA Security Design and Implementation, http://www.redbooks.ibm.com/abstracts/sg247310.html
WebSphere Process Server end-to-end security tutorial,
Audit trail in WebSphere Process Server, http://publib.boulder.ibm.com/infocenter/dmndhelp/v6r1mx/index.jsp?topic=/com.ibm.websphere.bpc.612.doc/doc/bpc/rg5attbl.html .
CICS Transaction Server example of a signed SOAP message,
Well here it comes again. Today I have been confronted to a project where the teams did not understand the need for variability and replaced the Characteristic Value variability pattern from the SID Telco model with static tags for each attributes.
At the end they make their interfaces signature rigid and will have to introduce new services for each different type of orders or products. This is typically what happend with Client-Server where changes on the server side propagated to the client side. We absolutely must avoid to use that Client-Server on Web Services approach for SOA as it ends up propagating and amplying the cost of changes to service consumers ,whether they are processes or other applications.
SOA needs to be about business or semantic loose coupling where the interfaces can absorb variations without having to change all of the consumers that use that specific interface.
The characteristic value pattern from the SID is as follows where the value point to a specification that defines the type, name and constraints. The information is structure in stable parts and variable parts with the variability of the information model enabled by this pattern.
An example of such variable message is also provided. The Characteristic values can be added to described very different aspects and do not change at all the structure of the messages even when adding new attributes to a product, service resource or a customer.
Have a nice day.
fiammante 100000A8UA Tags:  soa variability processes services information business 2,741 Views
It has been a long week since my last entry ;-) , publishing a book uses a lot of bandwidth.
Before going into variability, I want to explain the issue leading to variabiltiy
We have often forgotten that services exposed information that is carried by processes. Thus if the information evolves in its structure it will propagate to services and processes. Let me give you an example that I experimented in a project: A product catalog may contain attributes for these products. Whether these new attributes were represented or not as new columns in a database, they ended up being new tags in a schema such as <TV_Channels> for a video on demand product. Given that the schema changed the services accessing the product catalog changed signature, and the business processes using these services had to be regenerated and tested. Two product attribute changes per week were occuring, and the system test for the affected processes was 2 weeks on average per process. This ended being a catch 22.
That being said, I now need to answer to the following questions, how can I model the information to prevent structural changes and how can I evaluate the testing effort of a business process.
On the first question, I wrote a full chapter on various techniques in my book, including xsd:any I already mentionned, in this blog but the one we used in the specific case is the CharacteristicValue and CharacteristicSpec pattern from the Telemanagement Forum SID model for Telco operators. This model defines a characteristic specification that will describe the attribute with metadata, including the allowed values the type and validity dates. The characteristic value itself has a link the specification so that a common specification can be used for many values.
On the second question, the experience shows that creation, change and test effort of processes is roughly proportional to the number of arcs (connections) in a given business process. Even if you only change a small aspect you will need to test all internal variations. It is quite common to have two to four person hours of effort per arc in the process.
The following picture shows that with only 3 tasks and 5 nodes in a process you can have 10 arcs, so you may expect 5 days of test.
It somehow relates to a cyclomatic complexity used in software development test evaluation.
Another important aspect of variability are rules and policies. A consistent enterprise approach to rules and policies will require the creation of a common business vocabularity which content must be aligned with the concepts in the information model.
With OMG's SBVR there is now a standard for the structure of rules and policies but not of the contents which will always be specific to an industry and/or an enterprise. The vocabulary will describe the core elements of the information model, while the rules content model will define the acceptable value ranges when they are required by rules or policies.
If we now integrate this information variability with SOA and BPM but also with rules and policies we can have business processes which behavior is driven by the content of information and is much less sensitive to changes. Using a business vocabulary for the rules with a human language like rules or policy description enables business users to manipulate the rules and shift the changes from IT to business. In a further blog entry I plan to give real examples of such policies.
My regards to readers.
fiammante 100000A8UA Tags:  soa business_process business_process_manageme... variability 2,697 Views
What is the good ganularity of services ? Well I like to reformulate this question in what is the manageable granularity of services. How many services methods or interfaces can we manage in an Enterprise. If we take a decomposition like APQC Process Classification Framework at the task level which is the 4th level of decomposition, we get aroun 1,500 tasks for the cross industry elements. Each task would have several interfaces or methods. Looking at other decompositons like IBM's Component Business Modeling which is a two level decomposition or Telemanagement eTOM Business PRocess Framework we get an average of 7 to 10 elements at each level of decomposition. This would lead to a potential of between 10 thousand and 100 thousand interfaces at a level 5 of decomposition, which I think everyone would agree is not manageable.
The implication of that simple math is that a manageable granularity is between a level 3 and 4 of decomposition, and if you end up with finer interfaces, we need to consolidate into more variable paylod interfaces at a higher level. It also implies that a decomposition exists in the enterprise otherwise there is no way to find what is the decomposition level of a service.
In addition to the above mentionned CBM, APQC and eTOM, a method to find a tree decomposition from process variations is described by the article from MIT Sloan School of management A Coordination-Theoretic Approach to Understanding Process Differences.
As I mention in my book we have too much considered SOA as client/server on Web Services. We really need to think variability as a way to enable reuse and avoid the propagation of provider changes to consumers and vice versa. In complement to the approach for variability I describe in my book looking at information variability, service variability and process variability here are two good articles on variability that an architect I work with in a large european bank just sent to me as we are working on these topics together for that bank.
fiammante 100000A8UA Tags:  business_process_manageme... soa best_practices dynamic agility bpm 1 Comment 5,445 Views
As an IT architect currently in the Enterprise Architecture, Business Process Management and Services Oriented Architectures domains I sometimes feel as both a scout and an archeologist. I am from the generation that used punch cards punch tapes, manual entry of machine code with bit entry switches but also FORTRAN, All what we wondered in the early days of programming has become natural and easy. However it is essential to avoid the pitfalls and as I say good architects have scars but need to learn from the difficult cases. Being in a team doing real advanced projects in production with customers, we faced the necessity of delivering value with better and faster implementations, more manageable and flexible than the previous ones. I just completed writing this book where I capture these experience and lessons learned. Its publication due this summer by IBM Press and Pearson publishing. I start with a focused enterprise architecture approach, then look at methods to delivery variability, first addressing variability in information models, then service models and process models. I then follow with what I call the enterprise expansion joint, covering the enterprise services buses and additional practices of performing integration in a flexible way. I then complete by the tooling for the life cycle and the management and monitoring. This was an interesting effort and I developped all the models shown with the appropriate tools. The writing alone experience was also interesting, keeping the motivation. Communication is essential and often a good picture worth a thousand word, and I created quite a number of these pictures in the book including 3D pictures created with Google Sketchup . Here is one example of these pictures to show the mnemonic I created to remember the enterprise architecture layers that are, Business, Application and Services, Information and Infrastructure layers. To create such pictures you just need to create the planes with your usual graphing tool and export them as png that you can then import in Google Sketchup and position, with views and scenes of the same composition. In addition to such pictures, I also added to the text model examples and code examples which all were inspired from real projects but of course all written new by me for this book. I really hope that in really my readers will get immediate value for their projects, with this small brick I am adding to the IT industry.
I am from the generation that used punch cards punch tapes, manual entry of machine code with bit entry switches but also FORTRAN,
All what we wondered in the early days of programming has become natural and easy. However it is essential to avoid the pitfalls and as I say good architects have scars but need to learn from the difficult cases.
Being in a team doing real advanced projects in production with customers, we faced the necessity of delivering value with better and faster implementations, more manageable and flexible than the previous ones.
I just completed writing this book where I capture these experience and lessons learned. Its publication due this summer by IBM Press and Pearson publishing. I start with a focused enterprise architecture approach, then look at methods to delivery variability, first addressing variability in information models, then service models and process models. I then follow with what I call the enterprise expansion joint, covering the enterprise services buses and additional practices of performing integration in a flexible way. I then complete by the tooling for the life cycle and the management and monitoring.
This was an interesting effort and I developped all the models shown with the appropriate tools.
The writing alone experience was also interesting, keeping the motivation. Communication is essential and often a good picture worth a thousand word, and I created quite a number of these pictures in the book including 3D pictures created with Google Sketchup . Here is one example of these pictures to show the mnemonic I created to remember the enterprise architecture layers that are, Business, Application and Services, Information and Infrastructure layers.
To create such pictures you just need to create the planes with your usual graphing tool and export them as png that you can then import in Google Sketchup and position, with views and scenes of the same composition.
In addition to such pictures, I also added to the text model examples and code examples which all were inspired from real projects but of course all written new by me for this book.
I really hope that in really my readers will get immediate value for their projects, with this small brick I am adding to the IT industry.