Several folks have raised important points in this connection. Namely (assuming I have interpreted the comments correctly):
1. A question of SOA Governance:
"Do services come from some apriori (generalized, enterprise-wide or industry-wide) schema (Kant), or do they come from the specific local requirements (Hume)?"
2. Instill SOA into company DNA: "The DNA or "Nature" has to be "Nurtured" in an Information Technology (IT) still in its infancy. You could think of the IT DNA as the defined lower metrics required for any IT shop to deliver Information. The key performance criteria for Information delivery SOA as a roadmap to bridge the communications gap between the "IT think" people and the "Business driver think" people."
3. Natural Selection: The more we put in the DNA, the more restrictive the future generation of SOA will be, the less we put in the DNA the more fragile our SOA generation will be.
4. The Grand Design approach: "...grand design, and hints at a roadmap created by an external, pervasive entity ... the lingua franca for cross-platform coding...."
To me, project motivation comes from two very diametrically opposed sources: pragmatics of funding (business drivers) and the passion of professionals (to do "the right thing").
In some organizational cultures, we will have an entry level of maturity that is going to be project focused and natural selection may indeed rule.
At the other end of the spectrum we have neterprise wide transformation efforts that are occuring with some degree of SOA governance that follows a roadmap, "grand design". This allows the "instilling of SOA into Company DNA" in a spiral form : not necessarily top-down, not project by project, but by whichever comes first. The governance in place sees to it that where the passion of professionals or the pragmatics of funding are the drivers, that success is achieved within the designated , I repeat, designated scope.
As diverse projects weave their way to SOA, SOA governance sees to it that Variation-oriented Analysis is conducted across projects by the enterprise architecture board (with business and IT representation), Service Refactoring is done to factor out common services that eliminate redundancy (the third Service Litmus test).
Data Science, Machine Learning & API / SOA: Insights and Best Practices
Immanuel Kant, grandiose philosopher, maintained that we have "synthetic notions a priori"; notions prior to experience that are embedded, as we would perhaps say today, in our DNA. David Hume, the empiricist, on the other hand maintained that our mind is a "tabula rasa" -- a clean slate that experience writes upon and we have no notions prior to experiences.
Should SOA start to be embedded in the DNA of the Enterprise or is it something that should be left as best addressed piecemeal, project by project, on an "as need" basis?
What do you think?[Read More]
Adoption of SOA is a gradual process. While some organizations are dabbling into Web Services wrappering of applications as a means of exploring the world of SOA and deciding how to proceed, others are engaging in enterprise wide business transformation. Others are in between, and have defined their roadmaps, vision, strategy and criteria for assessment and governance. They are embarking on this journey by integrating application using Service-orientation. This latter is called service-oriented integration (SOI). This is the intermediate stage in which a lot of projects find themselves.
I have described this process and some patterns for embarking on this transformation to SOA using SOI .
Inside IBM, we have been developing and leveraging a Maturity Model and Process for SOA called the Service Integration Maturity Model (SIMM). This is designed to achieve desirable stages of maturity.
We have found it convenient to define seven levels of maturity that are defined by the level of de-coupling and flexiblity achievable at that stage of maturity:
1. Silo (data integration)
2. Integrated (application integration)
3. Componentized (functional integration)
4. Simple Services (process integration)
5. Composite Services (supply-chain integration)
6. Virtualized Services ( virtual infrastructure and integration through virtualization)
7. Dynamically Reconfigurable Services (eco-system integration)
Each have a detailed set of characteristics and criteria for assessment.
Here is an informal description of the highlights.
Level One. Organization start from proprietary and quite ad-hoc integration rendering their architecture brittle in the face of change.
Level Two. Organizations have moved towards some form of EAI (Enterprise Application Integration), albeit with proprietary connections and integration points. The approaches used have been to look at legacy systems and attempt to dissect and refactor through data integration.
Level Three. At this level, companies have componentized and modularized major or critical parts of their application portfolio. They have used legacy transformation and renovation methods to refactor legacy, expose functionality in a more modular fashion, built J2EE, or .NET based systems with clear component boundaries and scope.
The integration between components, is through their interfaces and the contracts between them.
Level Four. The company has embarked into the early phases of SOA by defining and exposing services for comsumption internally or externally for business partners; not quite on a large scale but acts as a Service provider none-the-less.
Level Five. Now the company has extended it's influence into the value chain and into the service eco-system. Services form a contract between suppliers, consumers and brokers that can build their own eco-system for on-demand interaction.
Level Six The company has now created a virtualized infrastructure to run applications. This is another level of decoupling achieved after the decoupled of the application, its servcies, components and flows. Now the infrastruture can be tuned and made agile through the notions of the grid and the grid service. Monitoring, management and events (e.g., common event infrastructure) have been externalized.
Level Seven This is a dynamically re-configurable software architecture. Services can be composed at run-time using externalized policy descriptions, externalized management and monitoring.
A simplified version of this has been discussed in 2003 in the Introduction I gave as guest editor of the Communications of the ACM on Enterprise Components and Services. Also, the four simplified levels have been discussed elsewhere as four adoption levels taken from SIMM .
Mappings to CMMI have also been done and are a natural way to try and understand maturity. The main challenge in a few of these efforts has been to be more than just a categorization according to some semblence of CMMI, and to justify why something is Defined, Repeatable or Optimizing, for example. This is often easy to do, due to the open and interpretable nature of the descriptions of each maturity level of CMMI.
The mapping to SIMM to CMMI is also quite straightforward; but the perceived value of this mapping seems distant other than the bandwagon factor of the success of CMMI. Service maturity relates to integration capabilities at various levels of business, methods, applications architecture and infrastructure.
CMMI deals with six levels of capability:
4. Quantitatively Managed
Each capability level consists of related specific and generic practices for a process area that can improve the organization processes associated with that process area.
Forrester has been discussing various notions of maturity, although from a different perspective.
Zapthink and Other analysts have discussed SOA roadmaps, which can be facilitated using SIMM.
Elements affecting maturity include coupling, standards usage, service identification that supports business needs, business models, goals and metrics that need to be supported by services, technologies, governance, infrastructure, tooling, skills and deployment capabilities.
Here is a developerworks article that provides more elaboration on this topic.
Ali_Arsanjani 120000D8QB 1,482 Views
Of course with the current hype around Web services and SOA, there are a number of events hovering around this topic.
IBM is holding a series of SOA Days . The main SOA page for IBM products and trends can be found here .
Another event is the 2nd SOA and Web Services Best-practices conference in Chicago, and another is a workshop we are holding around the same topic at OOPSLA 2005, for the third year.
One of the main attractions of these and similar events are the battle stories we can exchange around SOA, which in the final analysis makes all the difference: what works in practice is the more important aspect of software engineering for clients.[Read More]
Ali_Arsanjani 120000D8QB 2,565 Views
Summer vacation is over for most families with children. School has either started or is imminently going to... vacations are therefore usually over and we are back into the swing of things. We were talking about building the layers of an SOA, based on an SOA reference model. Let's discuss how to build the service component layer which is responsible for realizing the functionality and quality (adherence to non-functional requirements) of the service.
OK. So we assume know how to model and design an SOA. For that, we use some sort of Service-oriented Modeling method such as SOMA.
You have gone through Identification, Specification and Realization of Services, Service Components and Flows (processes).
During Realization you need to build the SOA. You need to map each service, service component and flow to a realization decision.
Once you have this table, you then have an idea of what you will:
3. Integrate (wrap, adapt)
4. Subscribe to (if event driven or publish and subscribe)
5. Transform (break apart a legacy system to get access to hitherto hidden nuggest of functionality not exposed in the orginal application)
Let's take number 1. Build. Building something is often about using a programming model to construct that piece of software.
In the 6th installation of the SOA Programming Model, Ferguson, Stockton and Nally talk about "A language-neutral, component-based programming model for service-oriented architecture (SOA) facilitates the implementation of Web services and their assembly into solutions."
This describes how to build the service components that will implement or realize your services.
The previous set of articles in that series can be found here.
There is some excitement with this rich client-side engine that mediates data interchange rather unobtrusively in a browser frame, catching input and sending it to the server, catching the results and more gracefully rendering the results. This is a step up from the staccato jerks of normal HTML page interaction, providing a smoother transition between almost unobtrusive loads.
Google's Maps or Google Suggest (see how the search narrows as you type) are examples of using this technology. You program this with things like ECMA Script .
Ajax is a composition of standards and technologies that includes:
* data interchange using (surprise!) XML
* data manipulation with good old XSLT
* asynchronous data retrieval using XMLHttpRequest
* standards-based presentation (XHTML and CSS)
* dynamic rendering and interaction using a tree structure similar to a Document Object Model
Adding this layer of mediation to the presentation layer of your SOA will further enable Web services consumption:
Ali_Arsanjani 120000D8QB 2,064 Views
A lot of times I am asked to describe and then implement an SOA as part of a client project.
One of the recurrent themes I have seen is that SOA can be described quite succinctly in terms of three dimensions:
functionality, policy and composition.
Functionality has to do with the fulfillment of requirements on the business side.
Policy has to do with the quality of service, non-functional aspects such as security, behavior in the face of changing levels of availability and duress that the SOA is put under.
Composition is in terms of how to combine the three key elements of SOA, namely, services, components and flows into more coarser-grained entities.
Ali_Arsanjani 120000D8QB 1,641 Views
National tennis associations tend to be run by local groups within communities that observe certain rules in their tournaments and are therefore sanctioned by the national tennis association. No matter how small you are, you can run a tournament if you can solicit enough participation from members and show a well-organized venue.
This lowers the barrier of entry and provides smaller cities the opportunity to stage tournaments and encourage skill in sports near a person;s hometown.
This network of sports communities are observing a common policy defined by a larger organization and implemented locally.
Similarly, as your company matures in its adoption and usage of SOA, and turns from small projects to larger and more intricate uses of SOA as depicted by the Service Integration Maturity Model (SIMM) , the need to provide servies to business partners in a value net and to leverage services provided by other partners in the same value net presents a unique opportunity and a set of challanges.
The interactions between parties are no longer one offs but rather collaborative goal-oriented interaction in whcih a participant alternatively assumes the role of provider, consumer or broker and enacts business transactions that run across not one company but several companies transparently.
The leveraging of the capabilities of other companies within thi service eco-system enables greater value than static partnerships or of relying solely on ones own resources.
Furthermore, participants in the SOA Eco-system can be small or larger companies and the smaller ones can be shielded from unnecessary judgements so lang as they maintain SLA's and provide the advertised functionality. This is a network effect that the Internet saw in its inception, but of a new caliber for businesses to leverage one another transparent to their customers. Great empowerment.
Ali_Arsanjani 120000D8QB 1,576 Views
I have been a strong advocate of the notion of service eco-systems and have been sharing patterns that help accomplish this task through the design of SOA and SOI that is conductive to the creation and maintenance of a stable service eco-system.
Others have been talking aout various aspects of a service eco-system as well. For example, The role of contracts in the service ecosystem . Some talk about the Web Services Eco-system and how we need tools for consumpiton, test, creation and management.
Recently, George Galambos of IBM delivered a keynote on the Service Eco-system at the IEEE Int'l Conference on Web Services 2005.
Aside from wildlife foundations, Economists also have found interest in ecosystems. For example, they discuss the valuation of an ecosystem .
We'll be hearing much more about this important concept as technologies and organizations mature in their usage and adoption of SOA.
Ali_Arsanjani 120000D8QB 1,417 Views
I have recently published an article here on dW regarding SOA and SOI patterns. Others have written in this space, most notably, Rachel Reinitz and Kyle Brown, and Bobby Wolf, three of my colleagues.
I focus on the ways in which we have done SOA with clients and discuss things that we see recurring, that seem to work well.
The Service Eco-system needs standards-based externalization of function, policy, context and events
Ali_Arsanjani 120000D8QB 2,498 Views
Well, over this weekend, deep in space, NASA has succeeded with Deep Impact and another impactor in the form of tennis balls were colliding: Federer has won his third Wimbledon in magnificent straight sets. As Roddick, his opponent, said,(paraphrase) "he is getting better than ever which is a surprise. I have been training hard and he is still getting better. maybe next time I'll punch him...."
The NASA mission showed our increasing precision through technology manipulated through software and (hardware)machines (many times context-aware services and components, but that is another story...) and the tennis champs showed they can progress with even greater precision on the human side.
Our precision in SOA and our success in winning with SOA depends not only on the initial standards and enabling technology, but on extending it to the enterprise scale and beyond into the ecosystem of partners.
To do this, we must focus on externalization of policy, of context and of sense and respond to events; whether the impact of a man-made satellite the size of a coffee table at 23,000 mph onto a raging bull of a comet, or Federer serving and acing at a precise 125 mph. These events, if signifcant to their corresponding businesses must be sensed and responded to. The response cannot be blind: a set of rules must be selected based on the context in which the event occurred and policies need to consulted in order to determine the valid/optimal course of action suitable for the business.
SOA seeks to externalize the interfaces to functionality, so a service consumer can find or use the URL/I and bind to the function without having to worry about the details of the underlying implementation; its design or technology. The service provider needs to also externalize policy information. Both provider and consumer need to monitor services and this is typically done via events. Events can serve as a service invocation mechanisms to wake up a service in a sense and respond scenario. And as the services are merrily firing on their own in blissful automation, the monitoring layering of an SOA should check for certian types of events that it is listening for based on business priority and policy.[Read More]
Ali_Arsanjani 120000D8QB 1,642 Views
I have been talking about the notion of a service eco-system for some time now. This is a mutually beneficial digital/human enviornment where actors are businesses that can choose to link to a remote resource using Web services. They combine their efforts in a value-net of services; similar to a supply chain, but not just for suppliers and consumers; but for companies who choose to increase their competative edge by joing forces with business services porvided by other participants in the service eco-system.
You need an SOA method to identify services that SHOULD be exposed to your eco-system.
Also there are various layers in your SOA. We call them a SOA Reference Architecure . One of these layers is the data or information management layer. Check out this recent article on aspects of information management for SOA .[Read More]
Ali_Arsanjani 120000D8QB 1,833 Views
One unique feature of SOA is that it is "fractal." The fractal nature of SOA implies that it is applicable in small projects, to one line of business, several lines of business sharing common services with slight variations, cross enterprise scope, between business partners in a value net and then on a larger scale in the service eco-system.
Ali_Arsanjani 120000D8QB 1,451 Views
Granularity is a tricky topic that I get queried about very often. It's more than a single metric measurement; it's how useful the interface is to you the potential consumer and how you can re-compose it in your own application context.
But there are many consumers out there; at one end of the spectrum are low level API's such as those typically found in packaged applications that vendors expose as Web Services. Then we get the grand bundling of an entire application as a service at the other end of the spectrum.
But it's all about how IT fits the business goals. Allow me to digress. In karate, there is a notion of "maai". It is the distance you have to step back to avoid being punched when the attacker comes in for a punch and the distance you have to step forward in order to be able to strike. 'So, how long is a "maai"?' someone may ask. I don't know how many feet and yes, it depends on ....
Back to SOA. When we do goal-service modeling, the composition of services and their granularity is attached to the sub-goal in the goal tree that you create. Higher level goals requiring services will have larger granularity services.
Granularity is inspired by buisness usage context that maps to business goals fulfillment that translates into what services you need to fulfill a given goal.
Okay, now, we will practice that move one more time. Yoi...(ready)...[Read More]
Of the six approaches to SOA, I briefly described the first one: top-down business driven. I am now thinking of how pilots are given various angles and approaches to airports based on traffic, wind, direction they are coming etc., which only shows that I need to get a life and travel less... but that's beside the point.
The others include: bottom-up legacy integration/enablement, data access driven, message or event driven , model-driven architecture (tools to generate web services) and legacy transformation.
Legacy Integration and Transformation are fundamentally bottom-up approaches: you start with something concrete and find a way to leverage it. Sorta like mathematical induction: from the particular to the general. Now back down to the ground of reality....
Customers want to expose functionality. In the first case functionality that is isolated and can be wrapped and mapped to a mid-tier application server. In the second case, the bottom-up approach is still related to legacy, but in this case the functionality requires some software archeology: it's embedded into the bowels of the legacy system. Now you need some form of compiler-based technologies (many of which evolved from the Y2K days) to extract re-usable portions of code that can be accessed from anywhere, not necessary through the mechanisms of the legacy system.
The two top down approaches are business driven and model-driven: the first one is based on bsuiness processes and the need to align and improve them. The second is based on tools and their ability to generate useful code from a top-down rendition of the business model or analysis model.
Some companies want to start with improving their businesses by focusing on the processes. Others may have more IT focus where they want to improve the business by improving the way IT does the job of supporting the existing processes.
Then we have two approaches that come in sideways: data access and message/event driven. Some people could care less about SOA or any other latest cool technology or promising paradigm; they just want to have a single uniform and consistent way to provide interface-based access to their underlying data: check the status of an order, place an order, access customer data, etc.
And others, coming in from the other sideways angle only care about integration; without knowing or caring about the two points of integration. In otherwords, I want to connect two systems they say and SOA gives me a very clean way to do so. I care only about messages exchanged between systems and the events that trigger the exchange of data or the invocation of a service.