SOA is the next step in software engineering. The service-oriented paradigm comes to an increasing degree of maturity roughly 5 years after what I consider to be the start of serious corporate adoption.
We migrated from procedural computing to object-based and object-oriented computing. Then we moved to improve the state of objects with component-based software engineering. Now, we have evolved our best-practices as an industry and in academia and have moved into the era of service-oriented computing.
The gradual adoption and service maturity can be seen to move along a spectrum we have depicted in the Service Integration Maturity Model.Here we have seven levels of maturity across 7 dimensions from infrastructure to the business architecture.
The implication of this paradigm shift is that the best-practices of service orientation need to be integrated into the fabric of software engineering to be utilized on projects that (this is important, so you may want to mute the TV for a second) SOA may either not be part of it or a very small part of the project. Still, you would use SOA PRINCIPLES such as using the service model for a specification of the required functionality within a project or an enterprise.
What are your views on the use of SOA principles on non-SOA projects?[Read More]
Data Science, Machine Learning & API / SOA: Insights and Best Practices
In the world of SOA, the economics of funding determines what services actually get funded to be developed. This is why we need a set of criteria by which we can assess which candidate services are a priority for inclusion in your next budget cycle. We call these criteria or gating factors, the Service Litmus Tests (SLTs). SLTs are included in SOMA , the de facto end to end SOA Development Method .
Not all candidate services should be implemented as services; we may not have enough budget for that. But here this: we STILL need the functionality of the services that fail to pass the SLTs. Therefore those services still need to be implemented as part of some service component or realized by an application.
Economics is hand in glove with governance and of course SOA Gov includes the issues of economics, funding and budgets and ownership.
Ali_Arsanjani 120000D8QB 6,999 Views
Lots of SOA projects later, I have finally gotten back to this blog.
I am seeing a large surge in SOA projects, each with greater maturity and more complex needs; but some of the basic remain the same. Often, projectsface unpredictable and complex human situations which may defy rigorous algorithms and require the soft art of consulting. Some are attempting to bridle this erratic and unpredictable aspect of human and group behavior. More of the SOA Projects later.
A recent post by the Univ of Arizona talks about the development of a software that sifts through tons of data and "will use sophisticated computational methods based on game theory, co-evolution and genetic development models to find solutions that make sense in illogical times. Genetic algorithms analyze situations in an evolutionary context, where actions with the highest “fitness factor” (chance of achieving the greatest success) gravitate toward one another, produce offspring and eventually rise to the top."
This form of evolutionary, convergent behavioral computing promises to be used in more and more simulation situations.Typically, rule engines are cluncky (technical term) and finding relations between multiple rules is a tricky proposition.Patterns can help. For example, the Business Rule Pattern Language I have documented starts with a lighteweight way of handling rules and moves into increasingly complex ways of handling rules within object oriented applications.
Follwing its use on three past projects, I have recently revised this pattern language to support SOA. I have gotten quite a bit of email requesting this and I am responding by saying that I will be publishing a draft soon.
Ali_Arsanjani 120000D8QB 6,947 Views
As people stood in line sometimes a day before the big day, June 29, I was thinking of the partnership between Apple and AT&T. What about other Service providers who provide wireless services; could they not interoperate and have a device, which theoretically might be provider agnostic work with their phone was well? Of course economics, partnerships and politics are the key driving forces here...
So in the world of SOA, economics, partnerships and politics play a significant role as well. In the world of SOA, the economics of funding determines what services actually get funded to be developed. This is why we need a set of criteria by which we can assess which candidate services are a priority for inclusion in your next budget cycle. We call these criteria or gating factors, the Service Litmus Tests (SLTs). They are embedded in SOMA , the de facto end to end SOA Development Method.[Read More]
Ali_Arsanjani 120000D8QB 6,864 Views
SOA Solutions are most often hybrid solutions. Yes, they focus on a set of services; but they often do not soley rely on services for the realization of the functionality that needs to be in place for the business.
SOA solutions tend to rely on a combination of architectural styles and implementation and realization constructs to craft the underpinnings of an SOA solution. The SOMA method utilizes a combination of approaches to Service Identification. This includes 6 perspectives: top-down business process driven, business policy and rule-driven approaches, bottom up legacy integration, bottom-up legacy transformation (intrusive changes to rip out legacy modules and expose them via access points), information as a service typically used to consolidate multiple backend data stores and resolve inconsistencies in the access, rules and synchronization of the data stores and lastly the message-driven approach which seeks to integrate systems using a service interface.
Among the approaches above, although some are more established than others, information as a service affords a unique perspective in solving challenges relating to information. For example, data access interfaces and their underlying data access logic might need to be externalized judiciously if multiple channels are seeking to access and manipulate data from potentially multiple access points. The need for consolidation, synchronization and management of the data along with the need to have a coherent set of policies be applied to the data calls for the information service “entry point” to SOA.
Service-oriented modeling and architecture (SOMA) has become the industry de facto standard for SOA Methods. Introduced by IBM in Jan 2005, released recently as RUP/SOMA 2.4, it covers the identification, specification, realization and implementation of services, components, flows (processes), information and composition.
SOMA uses Information Analysis, Modeling and Planning during identification, an Information Specification during design and a number of artifacts during Realization and Implementation including considerations for Enterprise Information, Master Data Management, Conceptual, Logical and Physical Data Models.[Read More]
[b]S[/b]ervice. Refers to how we use a service interface as a contract to decouple or loosely-couple a service provider from their prospective service consumers. Providers publish services, prefereably and increasingly in registries or service repositories (where they can be better governed). Consumers are looking for capabilities / functionality along with a set of non-functional requirements or service level agreements (SLA) that they want to be able to declaratively specify.
Services allow a business to expose its main operations to a well defined set of partners, clients and world at large, without giving direct access to its underlying IT systems, butthrough the surrogate of a web service.
The structure of an SOA, introduces services at an architectural level (a layer dedicated to services in the architecture).In object-oriented programming we tried to separate interfaces from implementation. IN SOA, we have carried that programming practice up to the architectural level and now have a layer dedicated to enforce that programming best-practice at a design level.
[b]O[/b]riented. The orientation is not object-orientation or component-orientation, but rather, service-orientation. This orientation is a tendency towards using services above the other options; not to exclude those options. Thus, when we apply the service litmus tests to the portfolio of candidate services we will end up with a smaller set of services and a number of other capabilities that still need to be implemented using some technology: legacy, package or custom, even if it is not a web services implementation.
[b]A[/b]rchitecture. SOA is a style of architecture and relies on the sound principles of software architecture, including the fact that it is merely one style, to be combined with other styles to form hybrid solutions to handle complex projects and real-world situations. Booch talks about bringing the 'A' back into SOA. I would say that as we overcome the hype of the acronym, we turn our attention to each letter and take action on what it signifies; including the A part for Architecture.[Read More]
[First a bit of history]. SOMA, IBM's SOA Method is now (officially) 2 years old. On Nov 9, 2004, I published a short paper describing IBM's SOA method: Service-Oriented Modeling and Architecture (SOMA) . We had been working on extending current methods for SOA before that, and I have documented our efforts in a SOA redbook (chapter four) which describes a primitive version of SOMA. Since then, a lot of work went into the method and SOMA took a quantum leap in 2004-2006 timeframe, as we formed teams around it and have successfully completed a very large number of projects in various industries and geographies and have taught about a thousand consultants and many clients to effectively use and deploy this method when designing SOA.Chances are that an SOA project will need to do a bit more than ad hoc web services implementation and need a bit of design. Some may even need some analysis of services components and flows, which are the fundamental elements of an SOA. Current methods do not have support for SOA and its fundamental constructs. This is where you can use SOMA, to identify, specify, design and realize the services, components and flows(processes) of your SOA. Based on a large number of project experiences over 2002-2004, we extended existing analysis and design methods, including global services method, which is an internal IBM proprietary method, as well as the rational unified process (RUP) and added the tasks and work products and roles necessary for the analysis and design and implementation of a service-oriented architecture (SOMA 2.x).
[Cut to present]. Yesterday, IBM announced a number of important tool updates. One of the accompanying plug-ins for the Rational Method Composer includes SOMA on top of the familiar industry method, RUP. Previously, SOMA was only available with IBM's internal Global Services (GS) Method. We have now extended this reach due to popular demand. This was a collaborative effort within IBM between Global Business Services (SOA and Web Services Center of Excellence's SOMA team) and Software Group's Rational division. Simon Johnston and I have been working to deliver SOMA on top of RUP. Not all tasks in a method have to do with services and SOA; so the underlying method, like RUP, can take care of more mundane tasks like use-case modeling, etc.The end result of this work is the release of IBM RUP for Service-Oriented Modeling and Architecture V2.4 , which represents the combination of the Rational Unified Process (including RUP for SOA) and IBM's proprietary method, SOMA. This means that IBM has a single commercial method for the development of SOA solutions, whether you buy that method for your own use or you contract with IBM services; the customer gets the value of the combined experience of IBM's product and services communities.
SOMA is maturing based on new trends and new requirements; so you can expect to see further releases of RUP/SOMA in the future.
It is convenient to enforce the notion of scope in an SOA: "exposed services" only make sense when you define the scope and context in which they will be exposed. We refer to SOA as being fractal. This means that you can apply SOA and expose services in a fractal manner: you can define services for a project, a LOB, a few LOB's , an enterprise, an eco-system. For example, a Service Portfolio (part of the Service Model) will have an attribute of scope that helps define, for example, how each business unit has it's own set of services they use "internally" and also a set of services they expose "externally" to other LOBs and the rest of the enterprise. Each scope can be a Service Provider and a Service Consumer.
This addition of scope and role to the service model alleviates many issues in governance, boundaries, funding and indeed in the identification and specification of services in your SOA.[Read More]
As the need for IT to absorb variations increases, with the demand for greater business flexibility,we are confronted with some basic questions: how do we design simple for today, to get the current job done, but not "box " ourselves in a corner so we can support the required flexibility ?I think one major answer to this problem is Variation-oriented Analysis and Design (VOAD). VOAD consists of three main types or axes of variation: structural (type|data), process and policy/rule variations. Structural variations are often based on the identification of Types: Customer Type has variations like Gold Customer, PLatinum Customer and Normal Customer. Often this relates to variations in the structure of the class, or of attributes and data associated with the entity (looking from both OO and Data views).Process variations are when you recognize that a Gold Customer may start from a common base, but branch out into a different set of activities for registration or loan processing, for example. Policy or Rule Variations relate to the Rules and Policies (Rules about rules) that apply for each Type of Customer, for example.Clearly these three aspects of VOAD are related and complementary. It is often useful, in practice to distinguish and treat these three axes of variation.Once you analyze the variations along each axis of variation, you come up with a set of variation points: things that will tend to change or remain less stable. Deal with each variation point by applying a pattern. For example if a variation point for calculation of interest is required for various types of Customers, then a Strategy Pattern would be used to handle / instantiate that variation point.Note that variations tend to occur across all layers of an SOA..Not all that changes is a variation that is warranted to capture and model: only those that are architecturally significant will be worth your while to consider. How do you tell? An architecturally significant variation is one in which impacts the architectural /design decisions you will make and have a trickle down effect that will influence subsequent decisions in how you will build your architecture (e.g., SOA). The "domino" that will alter the course of other decisions is a significant or relevant variation.ANother FAQ is why focus on variations? Variations are more difficult to handle than commonalities. Previous literature focused more on identification of commonality which IMO has less of an architectural ripple effect than understanding, isolating and externalizing variations.
What do you think?[Read More]
Ali_Arsanjani 120000D8QB 5,782 Views
After quite a lot time of blog-draught for me, travelling frenzy overtaking me, as I travel to my clients and to share best-practices on SOA with my colleagues around the world, I find that we are indeed reaching a new phase transition in SOA. Last year I saw many look for solutions that called for the first phase of service modeling, namely what I call Identification. The latter part of last year and this year has been more of the next phase: Specification (design of services components and flows). Now we are beginning to see more of Realization of SOA; including prototypes that are expanded and strengthened into gradually more robust and production systems that support service level agreements...[Read More]
Ali_Arsanjani 120000D8QB 5,148 Views
One of the interesting problems is not just whether to have governance, or more specifically, SOA governance, but how much of it's elements should be implemented over a period of the adoption of more advanced governance practices.
Clearly organizations are at different levels of maturity with regard to their need for, adoption of and implementation of SOA governance. Therefore, it becomes increasingly important to fit the degree of governance injected into the organition with the culture, priorities, implications of introducing control points and feedback; who takes care of the feedback and how it gets managed and fed back as actionable items into the system.
Using the Service Integration Maturity Model, we diagnose where an organization is with regard to six dimensions of maturity; one of which is the "organizational dimension" and includes governance. The target, desired state of maturity is analyzed and a roadmap to implement SOA governance for that targetstate of maturity is depicted. Think of a realse plan for software: it is more realistic to plan a set of releases than an all out implementation of all key features that the project is seeking to implement. It seems obvious in that context. So it is with SOA Governance.[Read More]
The Modeling and Design of an SOA should follow a method like SOMA (Service-oriented Modeling and Architecture). The Realization of your SOA will typically involve a hybrid approach that will include your legacy systems, possibly some packaged applications you own or intend to add to your portfolio and some custom applications you will seek to construct as you move your IT services forward to greater support of flexible business needs.
One of the challenges in this road to realization of an SOA is to know how to conduct Service Realization. Realization has to do with making Realization decisions about how you will be implementing the services, using which components, packages or legacy. It consists of:
1. Mapping Components to a SOA reference architecture2. Allocating services to components, packages or legacy required to realize them3. Conducting Technical Feasibilty explorations which defines a set of architecturally significant proof-of-concepts4. Making Realization Decisions
The following are some tips for making Realization Decisions:
1. Recognize that the realization WILL be a hybrid
2. Make realization decisions about which services (and their operations) will map to 2.1 a given existing asset (legacy), 2.2 which part will need to be custom built 2.3 which part will map to a packaged application (ISV package)
3. Do a gap-analysis for each of these choices. Most often, the mapping will not be complete: your new requirements will need "just a bit more" functionality or the mapping "will not quite cover the required functionality", or the legacy system was not built to handle the non-functional requirements imposed on the system.
4. Now that you have done a gap-analysis and decide there is a gap, you need to make some additional realization decisions on HOW you intend to bridge that gap; both from a technology view and a business-functional view. Will you use a special middleware product or build it yourself (inside your organization)?
5. The next major decision can be quite daunting: if there is a gap there are at least two ways to bridge that gap: change the business to suit the software or package or, customize and add to the package or existing software to meet the business needs. The old way of doing this is how packaged application vendors such as SAP, Oracle (Siebel), JD Edwards, etc. (in whatever state of acquisition they are right now)have approached the problem for years: change your business to fit the package from a process perspective. Customize the package from a data perspective to meet the organization's actual information needs (which should be pretty close to the templates given by the package vendors, by the way)
The SOA proposition is to put the business in the driver's seat; and not have the package vendors force changes to the business. A package-driven business vs a business driven IT. This is where methods such as SOMA can help with making these decisions in a judicious way. But there are no hard and fast answers here: only IT strategy coordinated with business strategy through SOA Governance is the key.[Read More]
Ali_Arsanjani 120000D8QB 5,939 Views
The Architecture section of develoeprWorks has released another Insight and Outlook column entitled: What is IT governance, and why should you care? Here are my views. I start by what I think IT governance consists of and how SOA governance relates to it. Then I describe what I have seen as the important aspects of governance that need to be taken into account. My observations are based on my interactions with clients and the issues they run into day-by-day. I think governance, specifically, SOA governance, is an "umbrella" that encompasses other assets and offerings such as SOA assessments, SOA Strategy and Planning , SOA Methods (such as SOMA), SOA Maturity etc. It lends them the oversight and support based on a set of agreed-upon enterprise policies that will ensure quality of service within the soa life-cycle.Also, Michael Liebow, VP of SOA within IBM Global Services describes his views on the state of maturity of SOA in the industry in an interview with SearchWebServices. Here is an excerpt relating to maturity, which is, in my opinion a governance issue. "SearchWebServices: Along those lines, how many companies have you run into that would earn a certificate of occupancy for their SOA? Liebow: Here's the deal, we've done thousands of implementations and nobody I know of has the full house built. We offer to the industry a maturity model for service-oriented architecture and there are seven levels of maturity within that. The first level of maturity starts around siloed applications. The notion though is that you get alignment between the business and IT, so that you have a business architecture, the application architecture, the infrastructure, the whole alignment from top to bottom in your organization. Level two speaks to the notion of EAI and essentially proprietary integration. Level three talks to the really decade-old notion of SOA, which was around components. These were not the same types of components we're talking about today, but around CORBA, COM, whatever. They were still somewhat hampered because they were hard-wired, but this is not a new concept. People have been trying to do this for a long time.Level four is where we get to services integration. It's Web services-based. You're able to expose services to be connected. We call it SOI, for services-oriented integration. It's an approach to integration that's much more flexible. And we think that most organizations we work with are trying to get to level four. The majority of organizations today are somewhere between one and three. Level five gets you to composite applications.SearchWebServices: Is it a rarity to see a company at level five these days?Liebow: You can see examples of level five and organizations that are starting to get there. They're real leaders in their industries. By no means would I say they have a full house, but they are pinpointing areas of the business where they want to build that capability. Yet the industry as a whole is just on the verge of touching this area. There are a number of startups that are providing aspects around this, but the major vendors don't really provide this.We think that the majority of the industry is just trying to get to level four. They are trying to articulate a vision around level five. Six and seven speak to a level of dynamic sense and response, automatic, autonomic systems that's a future state. No one's there to any significant degree."
Ali_Arsanjani 120000D8QB 5,913 Views
A previous version of the SOA Reference Architecture, Solution View was discussed in the Architectural Template section of Service-oriented Modeling and Architecture and in Bobby Wolf's recent posting on Composite Services .
We have found it important to add a few layers and make a lot of refinements. The meta-model describes each layer, the architectural building blicks in each layer, patterns of interaction and architectural decisions needed in each layer.
These layers are:
1. Consumer layer - any consumer of a service would reside in this layer
2. Business process layer (choreography and composition)-- a process uses a set of loosely coupled services in a choreography or composite application
3. Services layers -- a layer of service descriptions and policies, implemented through
4. Service Components -- (e.g., EJB's or .NET components) who privide the actual realization of the service operation, or the service directly uses or exposes
5. Operational Systems and Data -- which include packaged applications like SAP, Siebel, PeopleSoft (Oracle), Legacy systems, and of course the data bases that support the applications.
cross cutting these functional layers are the operational layers that support and intersect the above:
6. Integration Layer -- if you need/have an ESB it's here
7. Quality of Service layer -- all, aspects of security, monitoring, management and all other quality of service aspects are implemented and ensure through this layer
8. Data Architecture, Business Intelligence and meta-data layer -- provides data models, star cshemas or meta-data relating to and supporting the SOA
9. Governance Layer -- includes the procedures, processes, registry , repository and run-time governance needed to servce as support for the entire life-cycle
If you are interested in the details, and the above jives with what you are looking for, let me know. Comments welcome.[Read More]
Ali_Arsanjani 120000D8QB 5,320 Views
The Fibonacci sequence is a mathematical progression that starts with 0 and 1 and then continues to add numbers to this sequence that are equal to the sum of the previous two numbers. Thus, the first seven numbers in the sequence are: 0-1-1-2-3-5-8. People have started writing poems to this sequence since a blog posting on blogspot by a screenwriter/children's writer a couple of weeks ago. This is sort of a variation of the more restrictive haiku which usually consists of a pattern of 5, 7, and 5 morae, phonetic units (usually deemed to correspond to the syllables of a language). Poems often follow patterns, as a guiding element; loose standard or template.
In order to form and flourish, Service eco-systems also need to conform to patterns, to flourish with the implementation of their best-practices and with standards to provide them with general guidance, even if this is only to provide a means to extend or use only relevant parts of the standard in more realistic cirtcumstances. SOA Governance distributes policy for compliance with standards, inclduing acceptable extensions as well as templates ofr workproducts such as the Service Model, which includes the categorized Service Hierarchy among other elements.[Read More]
Ali_Arsanjani 120000D8QB 4,719 Views
Companies who want to collaborate and are in fact, business partners can do so in two ways: proprietary and standards-based.Standards-based is most flexible but covers a spectrum of industry standards such as ACORD for insurance to a standard XML Schema with pre-agreed tags. This is the spectrum of standards-based interaction. As we move from a few instances of collaboration and access to one another's business processes, the adoption of industry standards based on XML becomes more important: the rules of engagement are well-understood and have been well thought-out by a an industrty body or consortium. The downside of this is that often the industry standard is too broad and all-empassing making its use cumbersome and company's end up using only a fraction of the standard. Standardized message structures across Lines of Business within the enterpise and gradually across the eco-system become increasingly important.[Read More]
Ali_Arsanjani 120000D8QB 4,406 Views
I have picked one of the favorite topics that I have received most emails on; namely SOA Governance, and would like to expand it to the topic of SOA Governance for the Service Eco-system.
But let's start small: inside the organization. Within the organizational dimension, we have the need for governance and specifically SOA Governance.
To ensure that the SOA Life-cycle is carried out correctly, we need the support provided by SOA Governance.
SOA Governance is more like an umbrellla activity that 1)oversees the functioning of the SOA Life-cycle and 2)defines relevant, currently apropos policies, 3) inserts control points and 4) communicates these policies, the values expected at specific control points to 5) ensure compliance with those policies with artifacts produced by activities conducted within the life-cycle.
Governance is more declarative; management is executive. SOA Governance declares policies and activities it expects to see, and delegates to management to carry out these actions to ensure conformance to policies, or sometimes, more importantly, to determine why a given activity, project or artifact must obtain an exception and WHY IT SHOULD NOT CONFORM to the governance policies.
George Galambos and I gave a talk last week at the annual IBM Technical Leadership Exchange about a topic that we have been talking about for some time now: the service eco-system. If you want to build a service eco-system you may need to look at some patterns that help you build your service eco-system .
As we move towards greater maturity in the adoption of SOA,there are a number of key challenges that need to be overcome. I like to call them the Grand Challenges of SOA. The first 9 are:
1. Business Case for SOA
2. SOA Maturity and Roadmap: What do I do next?
3. Service-oriented Modeling and Architecture end-to-end
4. Industry Specific SOA
5. Building composite applications
6. Monitoring and managing across the eco-system
7. Governing the SOA eco-system
8. Eco-system flexibility with declarative policies, service management and externalized functionality.
9. Service proliferation challenges quality of service
What do you think?
Ali_Arsanjani 120000D8QB 3,844 Views
SOA Governance is about the governance of the three fundamental elements of SOA, namely, services, components and flows. So we are governing the process, the artifacts around services, components and flows. In the SOMA Method, the SOA Method used to model, analysis design services, components and flows, we identify, specify and realize these elements.
This relates to the monitoring of these elements throughout the life-cycle; putting in control and check points and policies around corrective action as we develop these.
Governance seeks to ensure adherence to/compliance with policy along the execution of a set of process steps that may start from the manual/human aspects of the life-cycle and continue onward into the runtime enviironment. It often accomplishes this goal by planning and instituting a set of check points or control points where process results are cross checked/validated with a set of standards (including permissible alternatives) as defined by policy.
SOA Governance sees to it that these elements are relevant to the organization (vitality), are being reviewed and validated by stakeholders and being communicated within the organization as the service model is being constructed within the life-cycle.
The governance of the service life-cycle is necessary to ensure that the needs of the business is supported by a set of flexibly re-composable IT services or components.
In order to do so, there needs to be policies, principles, checkpoints, reviews put into place from a process perspective; the execution of this is delegated to management Run-time governance of SOA on the other hand involves runtime monitoring of events and service execution to ensure compliance with the qualities of service declaratively defined by SOA policies.