BPM, APIs & Service-oriented Architecture: Insights and Best Practices
As the eco-system of business evolves into a more opportunistic convergence of business partners into an eco-system of value, similar to the notion of a value chain, but a more loosely coupled exchange of goods and services, the need is increasingly being felt for an upfront design of an architecture structure to enable smooth and measurable business operations.
Eco-system architecture defines the foundation of interaction between a set of semi-autonomous business entities that interact with each other using a set of configurable policies that define the valid business interactions among potential and prospective participants, such as business partners: providers, consumers and brokers.
Eco-system architecture extends Enterprise Architecture in a similar way that EA extends the notion of solution architecture. EA is analogous to "city planning" whereas ECSA is analogous to planning how the economy of the whole nation is structured through a set of rules, policies, standards and structures .
Even though SOA has been around for years, the community at large did not have a clear declaration of intent and "values", although individually, there appeared to be a large degree of consensus. Formalizing this consensus, a group of seasoned SOA practitioners got together recently and agreed to a high level declaration of intent.
Service orientation is a paradigm that frames what you do. Service-oriented architecture (SOA) is a type of architecture that results from applying service orientation.
We have been applying service orientation to help organizations consistently deliver sustainable business value, with increased agility and cost effectiveness, in line with changing business needs.
Through our work we have come to prioritize:
That is, while we value the items on the right, we value the items on the left more.
We follow these principles:
© 2009, the above authors, this declaration may be freely copied in any form, but only in its entirety through this notice.
We have just published the SOA Manifesto . This document helps guide the values and principles of service-orientation and service-oriented architecture.
"Service orientation is a paradigm that frames what you do.A set of 6 values are then outlined, followed by a set of 14 principles that help guide the implementation of the values.
When faced with the forces outlines in the value statements, we provide a preference for one rather than the other, even
though both may be viable under certain circumstances.
I will post links to other authors of the manifesto soon.
The IBM Service-oriented Modeling and Architecture(TM) (SOMA) is IBM’s end to end software development and integration method (aka, process life-cycle) based on service-oriented software engineering principles to produce enterprise and individual project-level SOA or non-SOA solutions. This method includes prescriptive guidance applicable during the phases and iterations required in a software development process life-cycle. This guidance includes the activities and tasks to be performed by various designated roles within the life-cycle requiring input work products and producing or updating output work products and deliverables at specified junctures within the life-cycle. The tasks leverage a set of techniques (“capability patterns”) that prescribe and promote the use of best-practices included in the method, in conducting and accomplishing the task and to instantiate the related work products and deliverables.
SOA enables business agility. It enables flexibility of IT systems that support the business architecture. As the business changes, the IT systems required to support the business in a volatile environment of competitive engagement, are less prone to change.
The history of software engineering started with the separation of functionality or processing and data. The data of the processes that manipulate the data were traditionally separate. There came a time when as the pendulum swung emphasis was placed more on the data in one era ended and the pendulum would swing and emphasis would be placed more on process. Eventually object oriented programming, which evolved into object-oriented design and object oriented analysis, played a key role in the unification of process and data. David Parnas was the first to suggest the notion of information hiding. The notion of information hiding was that access to information or access to data was strictly done through the functionality provided by an object. Those that object encapsulated its data and protected it from direct manipulation. It offered a set of operations that you could perform on the data, but you could only invoke those operations do not access the data directly.
Objects often reflected real-world objects. The identity of the real world object was used as an abstraction and implemented in a software system as a software object. Identity often corresponded to a real-world entity although helper objects evolve from the IT constructs necessary to implement the real-world construct in a computer system.
One of the most important principles of object oriented programming was based on a separation of concerns not only in the domain of abstraction but also in terms of separation of interface from implementation. Not only did we separate data from the operations that manipulated the data all in one object, but we did so one step further by separating the interface of the behavior from the actual implementation of that behavior. The behavior or rather its implementation, actually change the state of the object or as it were manipulated the data directly. However, there must be several ways in which we can implement this data manipulation. Even though there may be several ways in which we can implement this manipulation, there is typically one way to represent the interface or the externally visible signature that would be used by a consumer to request a change to be made to the data that the object owned.
This principle is called programming to interfaces rather than implementation.
These interfaces coupled with the notion of composing a set of similar objects that often had interdependencies among them into the next level of encapsulation which was called the component lead to a whole new era of -based development. It exported interface and could contain multiple objects all of which would typically be expected to be related to one another in some logically cohesive fashion. These objects were expected to be highly collaborative with one another and thus made sense to be called located within the packaging of a single component. This allowed not only manageable functionality but also decrease the risk that nonfunctional requirements would be violated.
The promise of object orientation
What is Cloud Computing? Similar to the Initial era in which SOA was trying to be defined, we are now in the era of more definitions. That is not in itself a negative thing, but a necessary part of the formation of new technology as they become mainstream.
Cloud computing is essentially an extension of SOA and service oriented computing. It provides concentrated infrastructure, platform and data center like capabilities offered at a set of services over the Internet. The cloud has several unique characteristics.
The cloud itself needs to be viewed from both a consumer and a provider perspective.
from the perspective of the consumer they are looking for a service-based delivery model over the web in which a highly reliable efficient self optimizing and feed your head up infrastructure capability are made available to them.
From a Provider perspective, the cloud involves two key aspects of self optimization and virtualization using a service-based delivery model.
The service-based delivery model can operate under one of the following kinds of services: platform as a service, software as a service an infrastructure as a service.
in order to connect the consumer with the provider, the provider must publish in some metadata standard information interchange format the quality of service that it can provide and the services that it can provide, so the service consumer can select among various provider of various services they provide and the various qualities of service that are offered for each type of service.
Could computing is fundamentally a service delivery model. It is instantiated by an architecture that provides the services and is consumed by a Service Consumer's own internal systems.
cloud computing is a
service delivery model
in which a virtualized set of
are offered by Cloud Service Providers with the choice of quality of service.
Service Consumers then have a choice
as to what provider to select
Further more, the model should support a seamless portable view of service such that services
from multiple providers can be choreographed by a consumer or broker without having to
know about the inner workings of the underlying cloud infrastructures.
What do you think?
Ali_Arsanjani 120000D8QB 3,825 Views
Amidst the current economic downturn, and in the mist of the survival instinct that is perculating around the world, it is tough to sit back and observe trends.But one unmistakable trend is that countries are shifting from an industry-focused economy (e.g., manufacturing, see the collapse of manufacturing toward a service economy. The impetus for this gradual change include globalization, technological change,and an overwhelming demand for (cheaper and cheaper) services. In observing this shift, it becomes increasingly clear that services and the service economy play an important role in today's and tomorrow's businesses. As a reflection of this trend, a realization or crystalization of the trend, service ecosystems have emerged: including GrandCentral, eBay, Google Base, Amazon.com, SalesForce.com, and SAP Business by Design to name a few. Such marketplaces enable trading products and also services between various legal entities and with consumers (sometimes referred to as the anonymous legal entity). One major challenge for service ecosystems is the fact that services are not products or goods. This has the potential to change the game.
Ali_Arsanjani 120000D8QB Tags:  service-orientedcomputing maturity pervasive services 4,235 Views
SOA continues to evolve through economic downturns and bends in the road. SOA has evolved into a set of underlying principles, concepts, technologies, paradigm and patterns of designing and building software; all predicated on a set of loosely coupled services that collectively support an organization's business processes and goals.These services provides the enterprise with an adaptive or agile capability to morph and transform itself in the face of environmental change.
As we look at the SOA Maturity Model we have called SIMM (Service Integration Maturity Model), we see a clear evolution path:
1. Simple Services (level 4.0 to 4.9) from 2005 -20072. Composite Services (level 5.0 to 5.9) from 2007 - 2009
now we see a lot of interest in paradigms and technologies that are BASED on Service principles, namely
3. SIMM level 6, Virtualized Services, in the form of Software-as-a-service (Saas) and Cloud Computing.
Services have become Pervasive, they can be found in many of the recent initiatives including Mashups, BPM, Infrastructure, Enterprise Architecture, Governance, etc.
The notion of an SOA is finding new applications in new areas and continues to morph and integrate into new things, ingesting new use-cases but retaining the base principles of service-oriented computing.
Ali_Arsanjani 120000D8QB Tags:  soa manners externalization voad variations voa 2 Comments 5,160 Views
In an effort to be responsive to a number of requests for an update to my old manuscript for "Principles of Advanced Software Engineering: Variation-oriented Analysis, Design and Implementation", let me give you folks some background on the subject. [Background]Originally, I wrote a manuscript back in 1999-2000 and used it to teach my classes of Graduate students in computer science for my advanced software engineering class. Since then I and a number of my colleagues have written several articles on this subject and have extended it into various domains: Variation-oriented engineering, variation-oriented requirements analysis, VOAD for SOA Solutions in addition to the base variation-oriented analysis and design. This topic also relates to my work on patterns for stability and symmetry in software architecture which I was writing for the pattern languages of programming conferences. I also did a blog entry on VOA that describes the high level basics of this paradigm.[More Current] VOAD is an integral part of the SOMA method for building end to end enterprise and applications solutions based on SOA principles (whether or not you have a full SOA implementation in mind. VOAD intersects with another very important notion that I have been researching and applying on projects, namely, context-aware services (CAS). Manners externalize semantics for on-demand composition of context-aware services.Manners define the context-aware behavior that is needed in order to externalize component manners to achieve greater maintainability and reuse. This article shows how you apply VOAD in order externalize the changing aspects of the software, so you can decrease maintenance costs, enhance reuse and actually increase flexibility in design and implementation of your software solution. This is a rather popular article and has been cited in a number of other publications . I think this is because we show the practical application of VOA to create a dynamically reconfigurable architectural style. Also, I believe this element is one of the characteristics that will be needed in more mature SOA Solutions. So as organization's adoption increases, they will need more mature ways of doing SOA, and VOAD is a key to achieving that.Empowering the business analyst for on demand computing explores Grammar-oriented object design (GOOD) which is the way in which we apply VOAD for externalizing manners. This article describes a project we did in the public sector for harmonization of multiple redundant business processes across the world. This is immediately reminiscent of the value that VOA brings: one of which is to consolidate business processes and optimize them by eliminating redundancy.I suspect that is why I am getting these emails on putting more out there on VOAD and to update my older work to reflect the impact and opportunities for SOA. I get the message :-) and will do so.
A new article in the IBM systems journal is out on SOMA that describes the method in a bit more detail than I did in 2004. This article describes a method that has now exercised hundreds of projects successfully and thousands trained worldwide in its delivery. This edition also features a number of articles on SOA.
Here is the abstract:
Service-oriented modeling and architecture (SOMA) has been used to conduct projects of varying scope in multiple industries worldwide for the past five years. We report on the usage and structure of the method used to effectively analyze, design, implement, and deploy service-oriented architecture (SOA) projects as part of a fractal model of software development. We also assert that the construct of a service and service modeling, although introduced by SOA, is a software engineering best practice for which an SOA method aids both SOA usage and adoption. In this paper we present the latest updates to this method and share some of the lessons learned. The SOMA method incorporates the key aspects of overall SOA solution design and delivery and is integrated with existing software development methods through a set of placeholders for key activity areas, forming what we call solution templates. We also present a fractal model of software development that can enable the SOMA method to evolve in an approach that goes beyond the iterative and incremental and instead leverages method components and patterns in a recursive, self-similar manner opportunistically at points of variability in the life cycle.
SOA is the next step in software engineering. The service-oriented paradigm comes to an increasing degree of maturity roughly 5 years after what I consider to be the start of serious corporate adoption.
We migrated from procedural computing to object-based and object-oriented computing. Then we moved to improve the state of objects with component-based software engineering. Now, we have evolved our best-practices as an industry and in academia and have moved into the era of service-oriented computing.
The gradual adoption and service maturity can be seen to move along a spectrum we have depicted in the Service Integration Maturity Model.Here we have seven levels of maturity across 7 dimensions from infrastructure to the business architecture.
The implication of this paradigm shift is that the best-practices of service orientation need to be integrated into the fabric of software engineering to be utilized on projects that (this is important, so you may want to mute the TV for a second) SOA may either not be part of it or a very small part of the project. Still, you would use SOA PRINCIPLES such as using the service model for a specification of the required functionality within a project or an enterprise.
What are your views on the use of SOA principles on non-SOA projects?[Read More]
In the world of SOA, the economics of funding determines what services actually get funded to be developed. This is why we need a set of criteria by which we can assess which candidate services are a priority for inclusion in your next budget cycle. We call these criteria or gating factors, the Service Litmus Tests (SLTs). SLTs are included in SOMA , the de facto end to end SOA Development Method .
Not all candidate services should be implemented as services; we may not have enough budget for that. But here this: we STILL need the functionality of the services that fail to pass the SLTs. Therefore those services still need to be implemented as part of some service component or realized by an application.
Economics is hand in glove with governance and of course SOA Gov includes the issues of economics, funding and budgets and ownership.
Ali_Arsanjani 120000D8QB 2,777 Views
Lots of SOA projects later, I have finally gotten back to this blog.
I am seeing a large surge in SOA projects, each with greater maturity and more complex needs; but some of the basic remain the same. Often, projectsface unpredictable and complex human situations which may defy rigorous algorithms and require the soft art of consulting. Some are attempting to bridle this erratic and unpredictable aspect of human and group behavior. More of the SOA Projects later.
A recent post by the Univ of Arizona talks about the development of a software that sifts through tons of data and "will use sophisticated computational methods based on game theory, co-evolution and genetic development models to find solutions that make sense in illogical times. Genetic algorithms analyze situations in an evolutionary context, where actions with the highest “fitness factor” (chance of achieving the greatest success) gravitate toward one another, produce offspring and eventually rise to the top."
This form of evolutionary, convergent behavioral computing promises to be used in more and more simulation situations.Typically, rule engines are cluncky (technical term) and finding relations between multiple rules is a tricky proposition.Patterns can help. For example, the Business Rule Pattern Language I have documented starts with a lighteweight way of handling rules and moves into increasingly complex ways of handling rules within object oriented applications.
Follwing its use on three past projects, I have recently revised this pattern language to support SOA. I have gotten quite a bit of email requesting this and I am responding by saying that I will be publishing a draft soon.
Ali_Arsanjani 120000D8QB 2,689 Views
As people stood in line sometimes a day before the big day, June 29, I was thinking of the partnership between Apple and AT&T. What about other Service providers who provide wireless services; could they not interoperate and have a device, which theoretically might be provider agnostic work with their phone was well? Of course economics, partnerships and politics are the key driving forces here...
So in the world of SOA, economics, partnerships and politics play a significant role as well. In the world of SOA, the economics of funding determines what services actually get funded to be developed. This is why we need a set of criteria by which we can assess which candidate services are a priority for inclusion in your next budget cycle. We call these criteria or gating factors, the Service Litmus Tests (SLTs). They are embedded in SOMA , the de facto end to end SOA Development Method.[Read More]
Ali_Arsanjani 120000D8QB 2,944 Views
SOA Solutions are most often hybrid solutions. Yes, they focus on a set of services; but they often do not soley rely on services for the realization of the functionality that needs to be in place for the business.
SOA solutions tend to rely on a combination of architectural styles and implementation and realization constructs to craft the underpinnings of an SOA solution. The SOMA method utilizes a combination of approaches to Service Identification. This includes 6 perspectives: top-down business process driven, business policy and rule-driven approaches, bottom up legacy integration, bottom-up legacy transformation (intrusive changes to rip out legacy modules and expose them via access points), information as a service typically used to consolidate multiple backend data stores and resolve inconsistencies in the access, rules and synchronization of the data stores and lastly the message-driven approach which seeks to integrate systems using a service interface.
Among the approaches above, although some are more established than others, information as a service affords a unique perspective in solving challenges relating to information. For example, data access interfaces and their underlying data access logic might need to be externalized judiciously if multiple channels are seeking to access and manipulate data from potentially multiple access points. The need for consolidation, synchronization and management of the data along with the need to have a coherent set of policies be applied to the data calls for the information service “entry point” to SOA.
Service-oriented modeling and architecture (SOMA) has become the industry de facto standard for SOA Methods. Introduced by IBM in Jan 2005, released recently as RUP/SOMA 2.4, it covers the identification, specification, realization and implementation of services, components, flows (processes), information and composition.
SOMA uses Information Analysis, Modeling and Planning during identification, an Information Specification during design and a number of artifacts during Realization and Implementation including considerations for Enterprise Information, Master Data Management, Conceptual, Logical and Physical Data Models.[Read More]
[b]S[/b]ervice. Refers to how we use a service interface as a contract to decouple or loosely-couple a service provider from their prospective service consumers. Providers publish services, prefereably and increasingly in registries or service repositories (where they can be better governed). Consumers are looking for capabilities / functionality along with a set of non-functional requirements or service level agreements (SLA) that they want to be able to declaratively specify.
Services allow a business to expose its main operations to a well defined set of partners, clients and world at large, without giving direct access to its underlying IT systems, butthrough the surrogate of a web service.
The structure of an SOA, introduces services at an architectural level (a layer dedicated to services in the architecture).In object-oriented programming we tried to separate interfaces from implementation. IN SOA, we have carried that programming practice up to the architectural level and now have a layer dedicated to enforce that programming best-practice at a design level.
[b]O[/b]riented. The orientation is not object-orientation or component-orientation, but rather, service-orientation. This orientation is a tendency towards using services above the other options; not to exclude those options. Thus, when we apply the service litmus tests to the portfolio of candidate services we will end up with a smaller set of services and a number of other capabilities that still need to be implemented using some technology: legacy, package or custom, even if it is not a web services implementation.
[b]A[/b]rchitecture. SOA is a style of architecture and relies on the sound principles of software architecture, including the fact that it is merely one style, to be combined with other styles to form hybrid solutions to handle complex projects and real-world situations. Booch talks about bringing the 'A' back into SOA. I would say that as we overcome the hype of the acronym, we turn our attention to each letter and take action on what it signifies; including the A part for Architecture.[Read More]
[First a bit of history]. SOMA, IBM's SOA Method is now (officially) 2 years old. On Nov 9, 2004, I published a short paper describing IBM's SOA method: Service-Oriented Modeling and Architecture (SOMA) . We had been working on extending current methods for SOA before that, and I have documented our efforts in a SOA redbook (chapter four) which describes a primitive version of SOMA. Since then, a lot of work went into the method and SOMA took a quantum leap in 2004-2006 timeframe, as we formed teams around it and have successfully completed a very large number of projects in various industries and geographies and have taught about a thousand consultants and many clients to effectively use and deploy this method when designing SOA.Chances are that an SOA project will need to do a bit more than ad hoc web services implementation and need a bit of design. Some may even need some analysis of services components and flows, which are the fundamental elements of an SOA. Current methods do not have support for SOA and its fundamental constructs. This is where you can use SOMA, to identify, specify, design and realize the services, components and flows(processes) of your SOA. Based on a large number of project experiences over 2002-2004, we extended existing analysis and design methods, including global services method, which is an internal IBM proprietary method, as well as the rational unified process (RUP) and added the tasks and work products and roles necessary for the analysis and design and implementation of a service-oriented architecture (SOMA 2.x).
[Cut to present]. Yesterday, IBM announced a number of important tool updates. One of the accompanying plug-ins for the Rational Method Composer includes SOMA on top of the familiar industry method, RUP. Previously, SOMA was only available with IBM's internal Global Services (GS) Method. We have now extended this reach due to popular demand. This was a collaborative effort within IBM between Global Business Services (SOA and Web Services Center of Excellence's SOMA team) and Software Group's Rational division. Simon Johnston and I have been working to deliver SOMA on top of RUP. Not all tasks in a method have to do with services and SOA; so the underlying method, like RUP, can take care of more mundane tasks like use-case modeling, etc.The end result of this work is the release of IBM RUP for Service-Oriented Modeling and Architecture V2.4 , which represents the combination of the Rational Unified Process (including RUP for SOA) and IBM's proprietary method, SOMA. This means that IBM has a single commercial method for the development of SOA solutions, whether you buy that method for your own use or you contract with IBM services; the customer gets the value of the combined experience of IBM's product and services communities.
SOMA is maturing based on new trends and new requirements; so you can expect to see further releases of RUP/SOMA in the future.
It is convenient to enforce the notion of scope in an SOA: "exposed services" only make sense when you define the scope and context in which they will be exposed. We refer to SOA as being fractal. This means that you can apply SOA and expose services in a fractal manner: you can define services for a project, a LOB, a few LOB's , an enterprise, an eco-system. For example, a Service Portfolio (part of the Service Model) will have an attribute of scope that helps define, for example, how each business unit has it's own set of services they use "internally" and also a set of services they expose "externally" to other LOBs and the rest of the enterprise. Each scope can be a Service Provider and a Service Consumer.
This addition of scope and role to the service model alleviates many issues in governance, boundaries, funding and indeed in the identification and specification of services in your SOA.[Read More]
As the need for IT to absorb variations increases, with the demand for greater business flexibility,we are confronted with some basic questions: how do we design simple for today, to get the current job done, but not "box " ourselves in a corner so we can support the required flexibility ?I think one major answer to this problem is Variation-oriented Analysis and Design (VOAD). VOAD consists of three main types or axes of variation: structural (type|data), process and policy/rule variations. Structural variations are often based on the identification of Types: Customer Type has variations like Gold Customer, PLatinum Customer and Normal Customer. Often this relates to variations in the structure of the class, or of attributes and data associated with the entity (looking from both OO and Data views).Process variations are when you recognize that a Gold Customer may start from a common base, but branch out into a different set of activities for registration or loan processing, for example. Policy or Rule Variations relate to the Rules and Policies (Rules about rules) that apply for each Type of Customer, for example.Clearly these three aspects of VOAD are related and complementary. It is often useful, in practice to distinguish and treat these three axes of variation.Once you analyze the variations along each axis of variation, you come up with a set of variation points: things that will tend to change or remain less stable. Deal with each variation point by applying a pattern. For example if a variation point for calculation of interest is required for various types of Customers, then a Strategy Pattern would be used to handle / instantiate that variation point.Note that variations tend to occur across all layers of an SOA..Not all that changes is a variation that is warranted to capture and model: only those that are architecturally significant will be worth your while to consider. How do you tell? An architecturally significant variation is one in which impacts the architectural /design decisions you will make and have a trickle down effect that will influence subsequent decisions in how you will build your architecture (e.g., SOA). The "domino" that will alter the course of other decisions is a significant or relevant variation.ANother FAQ is why focus on variations? Variations are more difficult to handle than commonalities. Previous literature focused more on identification of commonality which IMO has less of an architectural ripple effect than understanding, isolating and externalizing variations.
What do you think?[Read More]