Nasa has announced that one of its satellites have spotted a truly wonderful phenomena leading to the conclusion that it may have found evidence of liquid water reservoirs that erupt in Yellowstone-like geysers on Saturn's moon Enceladus.
After a long dry spell of blogging due mostly to heavy engagement on customer projects, I mark my midnight ruminations with the blog update. Multiple SOA projects in several industries: electronics, telecom and insurance. Sometimes I feel like what the satellite may feel like! Constant travel to meet with customers, find and fix their problems strategic and tactical in their journey towards SOA.
Like Cassini, I also discover some strange and wonderful things on client projects: the key challenges encountered on SOA projects. One of these is reklated to the human factor and the great influence of that factor versus the technical aspects of software engineering: the dynamics of organizations, people, communications, reporting, expectation setting and managing as well as mentoring others.
I play two roles: Chief Architect on client projects and university professor. As part of my latter duties I feel a need to convey some of my experiences to the next generation of software engineers that I teach in the following courses: Advanced Software Engineering, Dynamics of Software Architecture and Service-Oriented Architecture at the graduate level.
As I discuss how to become a software architecture with the masters in computer science I teach at MUM, one thing strikes me as very insightful to pass on to the students: that technology is many times the easy part! It seems to be the softer human skills that the architect needs to acquire for maximum effectiveness on SOA projects.
Data Science, Machine Learning & API / SOA: Insights and Best Practices
Ali_Arsanjani 120000D8QB 3,892 Views
Ali_Arsanjani 120000D8QB 3,535 Views
Many ISV's have professed new architectures for supporting SOA. For example , they are modeling typical business processes and exposing them as services.
Again, whether you are going down the SOA path by way of "engrain in company DNA", or "Natural Selection" or "Grand Design", you need to:
1. Integrate with legacy (operational systems layer), implies consolidation and multiple federated integration scenarios whihc would call for using best-practices in SOA design .
2. Ensure your consumer or presentation layer is loosely coupled
3. Then you can leverage all the SOA design sandwiched in between those two extremes.
4. BUT now, with the advent of the service eco-system , your company cannot maintain competative advantage unless is it at a maturity level that supports cross business partner interaction using, yes, you guessed right, services.
To create the eco-system, we need to understand what the business collaboration requirements of business partners are and what they are projected to be.
Often it's not only hard to obtain requirements, but also to get the business to say what they really want, in what order and by when; instead a general wand is waved(inbued with the most expensive peacock feather, as only those who hold purse strings can have).
But no matter what you gotta get those requirements translated into IT speak. The dW Architecture posed its second question: "How do I translate business needs into IT requirements?" I think it's mostly a matter of empowering business analysts with the ability and means to express their requirements properly and to help them do that, we need a teeny bit of rigor.
Increasingly, technology is helping make life easier in a large-scale, objective fashion albeit more focused in certain geographic regions. On the other hand, companis are also focusing on providing the consumer market with intensely subjective experiences which may indicate a trend of increased personal isolation: for example, a company has developed newwearable display for video iPods (typically costing 3x the cost of a typical iPod). We are accessible through cellphones, on the go; now we are accessible to entertainment providers, on-the-go.
Services on-the-go, tend to be context dependent; my choice in music and TV programs may be different from that of someone ten years younger. This may also be contingent on the time-zone I currently reside, to be able to get certain programming, and on the time of day. All of these point to a class of services, I have been referring to as context-aware services. An extreme version of this is when your cellphone announces to you that you are within walking distance of your favorite Chinese restaurant chain and it is near lunchtime in the current time zone.
Context-aware Services (CAS) is a convergence of a number technologies and paradigms: SOA, telecom, banking and personalization, to name a few. In some cases, context-aware services relate to the specific domain in which we are functioning: e.g., financial markets vs consumer banking vs insurance, etc.
This is what we will gradually see emerging on the horizon this year as the New Year dawns. Happy New Year!
Ali_Arsanjani 120000D8QB 3,827 Views
One of the important aspects of SOA Governance that we have been discussion is that of Service Domains: rather than having a lines of business own systems, Service Domain Owners choose Service Domains which group a set of related services. These affinity of these services are often of a business nature, or originating from a buisness perspective at the very least.
Decomposing a business domain into its sub-domains and functional areas enables a set of natural boundaries that can be a good starting point for Service Domains.
The Service-oriented Modeling Approach uses this strategy to identify domains, decompose them and then put governance around them.
Every domain owner will provide a set of services and wil in turn, itself require a set of services for consumption.
Thus the publication of the servcie descriptions of both these sets of services are critical to get the service eco-system going in your enterprise.
This allows the Domain Owners to make decisions about implementation changes to Service Implementations without changing the Servcie Description, if the change does not affect the SLA (service level agreement) and QoS (quality of service).
A set of base services can be instilled into an organization as the primary business functions it provides and decide which ones it in turn requires from its business partners. (the DNA approach we discussed earlier , recognizing the fact that there are three approaches to this paradigm, each of which may fit your enterprise.
Instill, let go and observe the evolution of services as they are adopted at various levels of maturity (or not), exercise governance for communication of the services, obtaining feedback to ensure they are relevant and to check for compliance.[Read More]
Back to the discussion of Service Domains: changes and ownership through governance.
Two readers have responded with interesting considerations.
MMahesh, your response brings about an important element of SOA: that of the separation of description from implementation and to IoD's point, if the change in question is only one of functionality and not SLA change or QoS change, then the Domain Owner has less of a say in this.
It thus appears that the service consumer expects a certain function (Service description) and a certain QoS. If these are satisfied, then the consumer if typiclaly happy.
Thus the change might not even affect the consumer, in which case the question is whether non-consumer affecting changes should be reported.
In some cases, I would like to point out that this may be warranted: where there is a deviance from a reference architecture, and i.e., of compliance. Now if the change makes sense then it must be propagated through the governance process of vitality which ensures continued relevance of the compliance criteria (eg a reference architecture or reference model) .
1. provider and consumer roles have their own concerns and governance applies to each individually and then to both to maintain a healthy service eco-system.
2. Change the implementation if you want (provider) or change it if you have to (consumer) and you are not getting your QoS or functionality. This ability to have the power to choose from various providers who suit your needs (functioanl and operational (QoS)), is a key feature of SOA.
3. Service Domains define a sphere of responsiblity, both functionally, operationally and financially.[Read More]
Now you have convinced people to invest in building services for flexibility, redundancy elimination and business/IT alignment. Great.
Each business line goes about their way to achieve it, but soon will find out that there is one conspicuous issue: who is the Service Owner? If I build this service and it is used in multiple places, who pays for the changes if the changes are those I do not specifically "care about" in my line of business?
Thus appears the notion of Service Domains, where a domain defines a set of related services that some "one" can own, maintain, support and (obtain) fund (ing).
To make a chance to a service, contact the Service Domain Owner.
Here is a question for you:
If I don't need to change the service description, can I change the service implementation? If so, do I still have to go to the service domain owner?
Several folks have raised important points in this connection. Namely (assuming I have interpreted the comments correctly):
1. A question of SOA Governance:
"Do services come from some apriori (generalized, enterprise-wide or industry-wide) schema (Kant), or do they come from the specific local requirements (Hume)?"
2. Instill SOA into company DNA: "The DNA or "Nature" has to be "Nurtured" in an Information Technology (IT) still in its infancy. You could think of the IT DNA as the defined lower metrics required for any IT shop to deliver Information. The key performance criteria for Information delivery SOA as a roadmap to bridge the communications gap between the "IT think" people and the "Business driver think" people."
3. Natural Selection: The more we put in the DNA, the more restrictive the future generation of SOA will be, the less we put in the DNA the more fragile our SOA generation will be.
4. The Grand Design approach: "...grand design, and hints at a roadmap created by an external, pervasive entity ... the lingua franca for cross-platform coding...."
To me, project motivation comes from two very diametrically opposed sources: pragmatics of funding (business drivers) and the passion of professionals (to do "the right thing").
In some organizational cultures, we will have an entry level of maturity that is going to be project focused and natural selection may indeed rule.
At the other end of the spectrum we have neterprise wide transformation efforts that are occuring with some degree of SOA governance that follows a roadmap, "grand design". This allows the "instilling of SOA into Company DNA" in a spiral form : not necessarily top-down, not project by project, but by whichever comes first. The governance in place sees to it that where the passion of professionals or the pragmatics of funding are the drivers, that success is achieved within the designated , I repeat, designated scope.
As diverse projects weave their way to SOA, SOA governance sees to it that Variation-oriented Analysis is conducted across projects by the enterprise architecture board (with business and IT representation), Service Refactoring is done to factor out common services that eliminate redundancy (the third Service Litmus test).
Immanuel Kant, grandiose philosopher, maintained that we have "synthetic notions a priori"; notions prior to experience that are embedded, as we would perhaps say today, in our DNA. David Hume, the empiricist, on the other hand maintained that our mind is a "tabula rasa" -- a clean slate that experience writes upon and we have no notions prior to experiences.
Should SOA start to be embedded in the DNA of the Enterprise or is it something that should be left as best addressed piecemeal, project by project, on an "as need" basis?
What do you think?[Read More]
Adoption of SOA is a gradual process. While some organizations are dabbling into Web Services wrappering of applications as a means of exploring the world of SOA and deciding how to proceed, others are engaging in enterprise wide business transformation. Others are in between, and have defined their roadmaps, vision, strategy and criteria for assessment and governance. They are embarking on this journey by integrating application using Service-orientation. This latter is called service-oriented integration (SOI). This is the intermediate stage in which a lot of projects find themselves.
I have described this process and some patterns for embarking on this transformation to SOA using SOI .
Inside IBM, we have been developing and leveraging a Maturity Model and Process for SOA called the Service Integration Maturity Model (SIMM). This is designed to achieve desirable stages of maturity.
We have found it convenient to define seven levels of maturity that are defined by the level of de-coupling and flexiblity achievable at that stage of maturity:
1. Silo (data integration)
2. Integrated (application integration)
3. Componentized (functional integration)
4. Simple Services (process integration)
5. Composite Services (supply-chain integration)
6. Virtualized Services ( virtual infrastructure and integration through virtualization)
7. Dynamically Reconfigurable Services (eco-system integration)
Each have a detailed set of characteristics and criteria for assessment.
Here is an informal description of the highlights.
Level One. Organization start from proprietary and quite ad-hoc integration rendering their architecture brittle in the face of change.
Level Two. Organizations have moved towards some form of EAI (Enterprise Application Integration), albeit with proprietary connections and integration points. The approaches used have been to look at legacy systems and attempt to dissect and refactor through data integration.
Level Three. At this level, companies have componentized and modularized major or critical parts of their application portfolio. They have used legacy transformation and renovation methods to refactor legacy, expose functionality in a more modular fashion, built J2EE, or .NET based systems with clear component boundaries and scope.
The integration between components, is through their interfaces and the contracts between them.
Level Four. The company has embarked into the early phases of SOA by defining and exposing services for comsumption internally or externally for business partners; not quite on a large scale but acts as a Service provider none-the-less.
Level Five. Now the company has extended it's influence into the value chain and into the service eco-system. Services form a contract between suppliers, consumers and brokers that can build their own eco-system for on-demand interaction.
Level Six The company has now created a virtualized infrastructure to run applications. This is another level of decoupling achieved after the decoupled of the application, its servcies, components and flows. Now the infrastruture can be tuned and made agile through the notions of the grid and the grid service. Monitoring, management and events (e.g., common event infrastructure) have been externalized.
Level Seven This is a dynamically re-configurable software architecture. Services can be composed at run-time using externalized policy descriptions, externalized management and monitoring.
A simplified version of this has been discussed in 2003 in the Introduction I gave as guest editor of the Communications of the ACM on Enterprise Components and Services. Also, the four simplified levels have been discussed elsewhere as four adoption levels taken from SIMM .
Mappings to CMMI have also been done and are a natural way to try and understand maturity. The main challenge in a few of these efforts has been to be more than just a categorization according to some semblence of CMMI, and to justify why something is Defined, Repeatable or Optimizing, for example. This is often easy to do, due to the open and interpretable nature of the descriptions of each maturity level of CMMI.
The mapping to SIMM to CMMI is also quite straightforward; but the perceived value of this mapping seems distant other than the bandwagon factor of the success of CMMI. Service maturity relates to integration capabilities at various levels of business, methods, applications architecture and infrastructure.
CMMI deals with six levels of capability:
4. Quantitatively Managed
Each capability level consists of related specific and generic practices for a process area that can improve the organization processes associated with that process area.
Forrester has been discussing various notions of maturity, although from a different perspective.
Zapthink and Other analysts have discussed SOA roadmaps, which can be facilitated using SIMM.
Elements affecting maturity include coupling, standards usage, service identification that supports business needs, business models, goals and metrics that need to be supported by services, technologies, governance, infrastructure, tooling, skills and deployment capabilities.
Here is a developerworks article that provides more elaboration on this topic.
Ali_Arsanjani 120000D8QB 3,502 Views
Of course with the current hype around Web services and SOA, there are a number of events hovering around this topic.
IBM is holding a series of SOA Days . The main SOA page for IBM products and trends can be found here .
Another event is the 2nd SOA and Web Services Best-practices conference in Chicago, and another is a workshop we are holding around the same topic at OOPSLA 2005, for the third year.
One of the main attractions of these and similar events are the battle stories we can exchange around SOA, which in the final analysis makes all the difference: what works in practice is the more important aspect of software engineering for clients.[Read More]
Ali_Arsanjani 120000D8QB 4,550 Views
Summer vacation is over for most families with children. School has either started or is imminently going to... vacations are therefore usually over and we are back into the swing of things. We were talking about building the layers of an SOA, based on an SOA reference model. Let's discuss how to build the service component layer which is responsible for realizing the functionality and quality (adherence to non-functional requirements) of the service.
OK. So we assume know how to model and design an SOA. For that, we use some sort of Service-oriented Modeling method such as SOMA.
You have gone through Identification, Specification and Realization of Services, Service Components and Flows (processes).
During Realization you need to build the SOA. You need to map each service, service component and flow to a realization decision.
Once you have this table, you then have an idea of what you will:
3. Integrate (wrap, adapt)
4. Subscribe to (if event driven or publish and subscribe)
5. Transform (break apart a legacy system to get access to hitherto hidden nuggest of functionality not exposed in the orginal application)
Let's take number 1. Build. Building something is often about using a programming model to construct that piece of software.
In the 6th installation of the SOA Programming Model, Ferguson, Stockton and Nally talk about "A language-neutral, component-based programming model for service-oriented architecture (SOA) facilitates the implementation of Web services and their assembly into solutions."
This describes how to build the service components that will implement or realize your services.
The previous set of articles in that series can be found here.
There is some excitement with this rich client-side engine that mediates data interchange rather unobtrusively in a browser frame, catching input and sending it to the server, catching the results and more gracefully rendering the results. This is a step up from the staccato jerks of normal HTML page interaction, providing a smoother transition between almost unobtrusive loads.
Google's Maps or Google Suggest (see how the search narrows as you type) are examples of using this technology. You program this with things like ECMA Script .
Ajax is a composition of standards and technologies that includes:
* data interchange using (surprise!) XML
* data manipulation with good old XSLT
* asynchronous data retrieval using XMLHttpRequest
* standards-based presentation (XHTML and CSS)
* dynamic rendering and interaction using a tree structure similar to a Document Object Model
Adding this layer of mediation to the presentation layer of your SOA will further enable Web services consumption:
Ali_Arsanjani 120000D8QB 4,186 Views
A lot of times I am asked to describe and then implement an SOA as part of a client project.
One of the recurrent themes I have seen is that SOA can be described quite succinctly in terms of three dimensions:
functionality, policy and composition.
Functionality has to do with the fulfillment of requirements on the business side.
Policy has to do with the quality of service, non-functional aspects such as security, behavior in the face of changing levels of availability and duress that the SOA is put under.
Composition is in terms of how to combine the three key elements of SOA, namely, services, components and flows into more coarser-grained entities.
Ali_Arsanjani 120000D8QB 3,708 Views
National tennis associations tend to be run by local groups within communities that observe certain rules in their tournaments and are therefore sanctioned by the national tennis association. No matter how small you are, you can run a tournament if you can solicit enough participation from members and show a well-organized venue.
This lowers the barrier of entry and provides smaller cities the opportunity to stage tournaments and encourage skill in sports near a person;s hometown.
This network of sports communities are observing a common policy defined by a larger organization and implemented locally.
Similarly, as your company matures in its adoption and usage of SOA, and turns from small projects to larger and more intricate uses of SOA as depicted by the Service Integration Maturity Model (SIMM) , the need to provide servies to business partners in a value net and to leverage services provided by other partners in the same value net presents a unique opportunity and a set of challanges.
The interactions between parties are no longer one offs but rather collaborative goal-oriented interaction in whcih a participant alternatively assumes the role of provider, consumer or broker and enacts business transactions that run across not one company but several companies transparently.
The leveraging of the capabilities of other companies within thi service eco-system enables greater value than static partnerships or of relying solely on ones own resources.
Furthermore, participants in the SOA Eco-system can be small or larger companies and the smaller ones can be shielded from unnecessary judgements so lang as they maintain SLA's and provide the advertised functionality. This is a network effect that the Internet saw in its inception, but of a new caliber for businesses to leverage one another transparent to their customers. Great empowerment.
Ali_Arsanjani 120000D8QB 3,587 Views
I have been a strong advocate of the notion of service eco-systems and have been sharing patterns that help accomplish this task through the design of SOA and SOI that is conductive to the creation and maintenance of a stable service eco-system.
Others have been talking aout various aspects of a service eco-system as well. For example, The role of contracts in the service ecosystem . Some talk about the Web Services Eco-system and how we need tools for consumpiton, test, creation and management.
Recently, George Galambos of IBM delivered a keynote on the Service Eco-system at the IEEE Int'l Conference on Web Services 2005.
Aside from wildlife foundations, Economists also have found interest in ecosystems. For example, they discuss the valuation of an ecosystem .
We'll be hearing much more about this important concept as technologies and organizations mature in their usage and adoption of SOA.