Data Science, Machine Learning & API / SOA: Insights and Best Practices
[First a bit of history]. SOMA, IBM's SOA Method is now (officially) 2 years old. On Nov 9, 2004, I published a short paper describing IBM's SOA method: Service-Oriented Modeling and Architecture (SOMA) . We had been working on extending current methods for SOA before that, and I have documented our efforts in a SOA redbook (chapter four) which describes a primitive version of SOMA. Since then, a lot of work went into the method and SOMA took a quantum leap in 2004-2006 timeframe, as we formed teams around it and have successfully completed a very large number of projects in various industries and geographies and have taught about a thousand consultants and many clients to effectively use and deploy this method when designing SOA.Chances are that an SOA project will need to do a bit more than ad hoc web services implementation and need a bit of design. Some may even need some analysis of services components and flows, which are the fundamental elements of an SOA. Current methods do not have support for SOA and its fundamental constructs. This is where you can use SOMA, to identify, specify, design and realize the services, components and flows(processes) of your SOA. Based on a large number of project experiences over 2002-2004, we extended existing analysis and design methods, including global services method, which is an internal IBM proprietary method, as well as the rational unified process (RUP) and added the tasks and work products and roles necessary for the analysis and design and implementation of a service-oriented architecture (SOMA 2.x).
[Cut to present]. Yesterday, IBM announced a number of important tool updates. One of the accompanying plug-ins for the Rational Method Composer includes SOMA on top of the familiar industry method, RUP. Previously, SOMA was only available with IBM's internal Global Services (GS) Method. We have now extended this reach due to popular demand. This was a collaborative effort within IBM between Global Business Services (SOA and Web Services Center of Excellence's SOMA team) and Software Group's Rational division. Simon Johnston and I have been working to deliver SOMA on top of RUP. Not all tasks in a method have to do with services and SOA; so the underlying method, like RUP, can take care of more mundane tasks like use-case modeling, etc.The end result of this work is the release of IBM RUP for Service-Oriented Modeling and Architecture V2.4 , which represents the combination of the Rational Unified Process (including RUP for SOA) and IBM's proprietary method, SOMA. This means that IBM has a single commercial method for the development of SOA solutions, whether you buy that method for your own use or you contract with IBM services; the customer gets the value of the combined experience of IBM's product and services communities.
SOMA is maturing based on new trends and new requirements; so you can expect to see further releases of RUP/SOMA in the future.
Architectural Paradigms of the Future Part 2: Eco-system Architecture engulfs Enterprise and Business Architecture in the Cloud
Ali_Arsanjani 120000D8QB Tags:  service-oriented-architec... enterprise-architecture business-architecture eco-system-architecture 14,597 Views
In the context of a cloud formation, where is the "enterprise", per se? Of course it exists in the traditional sense. And so does enterprise architecture. But in the context of a cloud, which may span private, public and hybrid instances, where is the "enterprise" now?
Rather than a physical location, the enterprise now seems to be more nebulous and "logical" or "conceptual". It now can be seen to expand possibly, with the cloud, as the amorphous mass expands so does the virtual enterprise. This is where the enterprise merges with the eco-system and you see the emergence and need for addressing aspects of the eco-system that span physical and logical enterprises. As we needed enterprise architecture, for the cloud we need an eco-system architecture.
Eco-systems are more dynamic, opportunistic in some instances; some shorter and some longer enduring alliances between interested business parties. This not only affects the IT side, as it were, in a "bottom up" fashion, but also the business architecture; in a more "top down" manner.
The structure and function of a business is defined by its model of what goods and services it provides and consumes; who it needs to partner with at what point in its dynamic existence. This relates to policies that cut across and senses the eco-system for more appropriate alliances and takes competitive advantage of this new configuration. A business architecture in an eco-system architecture is then configured by rules and policies driven by competitive business goals, not statically defined.
The cloud provides a virtual rockbed for the reconfigurable enterprise that supports its dynamic business architecture to sense and respond, anticipate and adapt to the ebb and flow of trends in the eco-system. The eco-system architecture defines and configures these elements that include the traditional business architecture, multiple enterprise architecture plus a set of rules, policies, that define desirable (to be sought) and undesirable (to be avoided) configurations of the business and IT as they ride on top of the virtual infrastructure provided by the cloud.
This constitutes the eco-system architecture of the future.
When Service identification Becomes Critical: Service Proliferation Syndrome and Service Interface Changes
Ali_Arsanjani 120000D8QB Tags:  governance service_portfolio service_proliferation service_model service_monitoring soma 13,779 Views
As our industry continues to mature in the adoption of SOA and related technologies, we are witnessing not only the subsiding of hype but also, the emergence of new issues that are surfacing due to growing pains. These issues include the need to change service interfaces as they evolve in conjunction with changing business requirements. The service portfolio which is the single point of reference for an enterprise's business capabilities undergoes evolution with external and internal changes that are ongoing within and without the enterprise.
As the Service Portfolio, part of the Service Model, matures, changes to the service interface, and the service contracts are inevitable. I was approached with one solution: to put them under change control. However expedient this measure, it is not a cure, but temporary stabilizers of organizational turbulence. Changes to the underlying IT capabilities that reflect the business needs that are dealt with in a proactive manner tend to stabilize over time and yield the agility that helps power sustained business performance.
Properly designing the services, i.e., proper Service identification techniques and practices are paramount for alleviating these types of issues. Second best is to use a registry and repository to store and manage them centrally. And of course you need a source code control system to manage the code that is going to be using the services.
The use of correct Service identification for alleviating the risks and impact of Service Interface and Service Contract changes coupled with management and governance in a Service Registry and Repository are thus paramount.
Further, as services abound and the organization feels more comfortable in the creation of new or modified (see above) services, the need to manage and govern this Service Proliferation Syndrome becomes increasingly important. Again there are two aspects : one of which is doable at design time and another at runtime. Service Identification coupled with Service Refactoring and Rationalization (SRR) help mitigate the first aspect -- design time best-practices. Secondly, the monitoring of Services at runtime come hand in hand with the Service identification and Monitoring approach to alleviate Service Proliferation.
The short answer is:
Cloud is an initial application of SOA to infrastructure
--its setup, configuration, monitoring , management , with the essential elements of metering and billing added to satisfy the goal of a charge model of e.g., pay-as-you-go
-- the specific requirements for XaaS: resource pooling and virtualization , elasticity and multi-tenancy, dynamic configuration and provisioning
Cloud paradigm leverages SOA to deliver a charge-based model for non-business specific services;
Several people have written articles on this relationship:
A new article in the IBM systems journal is out on SOMA that describes the method in a bit more detail than I did in 2004. This article describes a method that has now exercised hundreds of projects successfully and thousands trained worldwide in its delivery. This edition also features a number of articles on SOA.
Here is the abstract:
Service-oriented modeling and architecture (SOMA) has been used to conduct projects of varying scope in multiple industries worldwide for the past five years. We report on the usage and structure of the method used to effectively analyze, design, implement, and deploy service-oriented architecture (SOA) projects as part of a fractal model of software development. We also assert that the construct of a service and service modeling, although introduced by SOA, is a software engineering best practice for which an SOA method aids both SOA usage and adoption. In this paper we present the latest updates to this method and share some of the lessons learned. The SOMA method incorporates the key aspects of overall SOA solution design and delivery and is integrated with existing software development methods through a set of placeholders for key activity areas, forming what we call solution templates. We also present a fractal model of software development that can enable the SOMA method to evolve in an approach that goes beyond the iterative and incremental and instead leverages method components and patterns in a recursive, self-similar manner opportunistically at points of variability in the life cycle.
Even though SOA has been around for years, the community at large did not have a clear declaration of intent and "values", although individually, there appeared to be a large degree of consensus. Formalizing this consensus, a group of seasoned SOA practitioners got together recently and agreed to a high level declaration of intent.
Service orientation is a paradigm that frames what you do. Service-oriented architecture (SOA) is a type of architecture that results from applying service orientation.
We have been applying service orientation to help organizations consistently deliver sustainable business value, with increased agility and cost effectiveness, in line with changing business needs.
Through our work we have come to prioritize:
That is, while we value the items on the right, we value the items on the left more.
We follow these principles:
© 2009, the above authors, this declaration may be freely copied in any form, but only in its entirety through this notice.
Ali_Arsanjani 120000D8QB Tags:  bpm rest soa analytics microservices mobile cloud big_data api patterns 8,018 Views
Microservices reminds us of very fine grained, tiny, micro-scopic services. Perhaps services that grow on a Petri dish.
But actually, they refer to a deployment pattern for services developed in a Service-oriented Architecture along the constraints of a Functional Area that owns its own Data, and does not allow access to its Information willy nilly. You start with a domain break it down into a Functional Area and get down to the subsystems and components living there, deep in the bowels of the legacy systems, the packages inhabiting the digital ooze in the sewers of legacy code , interspersed with some fresh sprinklings of new additions of components implementing newer services required by the business.
Stratified along a Functional Area, immune to the curious and invasive glances of strange components and outside systems, the service rests peacefully in isolated, independent deployment. Choosing to deploy to brew this SOA flavor with a twist of independent deployment, along the lines of a functional area, owning its information, is detailed out in the implementation of finer grained services in SOMA (Service-oriented modeling and architecture) ,many companys' service-oriented method.
Don't mistake this as the only way to choose to constrain SOA by design along functional area boundaries and deploy along those lines. This is only one brew of SOA.
Some considerations are the need to communicate between deployed services that are other wise feeling a little isolated and engaged heads down in their servicing of requested for their functional area of the business. But business have cross cutting concerns and dimensions, often exemplified by BPM or business process management. Cutting across silos or functional areas, the process needs access to various data sources and systems of record, albeit some are prehistoric gauging by enterprise standards.
Sp cross cross cuts of SOA flavor are best served by BPM. Smaller granularity focused functional area services by Microservices. What about when mobile needs to access something? Well for that we have the API brew. RESTful (Representational State Transfer) APIs provide an HTTP/S level of simplicity of access (yes, both get and set) that allow access to underlying functionality via mobile devices and yes Software-as-a-services, or Cloud as is known in slang.
So choose your brew of SOA according to your mood, according to what you would, contingent on what you should
be doing with your processes, data or business functions. Choose wisely for each have pluses and minuses; opportunities and consequences, like any pattern. Or brew.
Ali_Arsanjani 120000D8QB 7,747 Views
I've participated in a lot of SOA projects around the world and very often I'm asked to answer questions about aspects of SOA. These are often recurrent questions that most of my clients and her colleagues are asking. So a colleague of mine, Kerrie Holley and I decided to consolidate these questions and answers into a book so that more people can access the questions, see if they are relevant to their particular circumstances and hopefully benefit from the responses.
The book is entitled 100 SOA Questions: Asked and Answered .
I have started a separate blog at soaquestions.blogspot.com to discuss further questions and comments specifically about the book .
Ali_Arsanjani 120000D8QB 7,709 Views
Over the years as I have taught, designed and implemented software architecture the question I have been asked over and over again has been:
What is Architecture?
According to the Arsanjani view of Architecture :-), the Architecture of a system is the holistic view of the relative configuration of a set of static and dynamic elements that include what I call the 6C's :
Components -- what are the structural building blocks of a solution, or a style of architecture, of the elements that can be combined to produce a larger structure
Connectors -- The components need to connect with one another, whether statically (as in an Entity-relationship kind of relation) or dynamically, as in a composition in an SOA, where you may have orchestration or choreography.
Constraints -- The constraints on the connectors and/or components that provide rules of engagement of what is permissible and what is not.
Composition -- How to compose or what are the valid compositions of components
Context -- the context of invocation of a component is critical to the designation of how that component will behave
Containers -- the components must live in some runtime container that will provide uniform life-cycle management capabilities for them
Traditionally, the first three have been the main focus in universities and textbooks. But the latter three that I have added seem to be essential to a more complete depiction of the operational or actionable perspective on software architecture.
Let me know what you think.
it is important to source IT constructs that directly support the business goals and processes. In order to do so we will explore the six fundamental constructs of service-oriented architecture. Elevation of the notion of the service or service interface and later on the service contract from a programming construct to the level of an architectural abstraction was a key paradigm shift in software engineering. By the service does not stand alone. It needs to be implemented using some component. The data has to be passed to the service to the component back to the databases and data warehouses and information systems in the backend, processing needs to be completed information retrieved and returned to the requester of the service. Thus information is paramount for services. As a service proceeds with its processing there are certain rules that apply. The invocation of the service should be a more intelligent invocation than that of the method on an object. Generally speaking it is common practice to invoke a method on an object without the necessary cards and without the necessary application of rules. It is typically left up to the application logic to implement those rules. With service-oriented architecture is important to although externalize the rules pertaining to a service but make the application of those rules visible to the service consumer or service requester. The service is also governed by a set of policies that allow the service to be changed without having to fundamentally change the implemented capability behind the service. Services should not only respond to a direct invocation but should be able to respond automatically in a subscriber will fashion to the triggering of that even
Thus it is critical to understand the six fundamental constructs of service-oriented architecture: services, processes, components, information, rules and policies and events. Services can be a comic or composite. As they participate as the steps of the business process, services can be invoked at each one of these steps allowing a shared set of common services to be utilized in a BPM context. therefore, processes are also one of the fundamental constructs of SOA.
Your architectural description should capture these six constructs as you meander through the journey of the software development lifecycle.
also, note that all of the six constructs are motivated by business level constructs. The following diagram establishes a relationship tween the business constructs and the corresponding SOA paradigm constructs.
Ali_Arsanjani 120000D8QB Tags:  sustainable adaptive performance agile business_performance business 7,569 Views
Following the thread on Agile and Adaptive Business Performance and Optimization
Business Performance (BPer) is a measure of key performance indicators over a period of time. Within that period, we indicate there is BPer if the anticipated metrics and KPIs are exceeded by the actual KPIs.
For this to happen, Business Processes need to have been identified and engineered appropriately to achieve business goals measured by the KPIs.
The underlying Services that are orchestrated to produce the Business Processes should be selected or identified using a rationalized Service Identification method
(e.g., SOMA -- Service Oriented Modeling and Architecture) that encompasses the needs of various aspects of granularity, performance, agility, flexibility and complexity.
The Business Architecture should follow an agile and adaptive model to incorporate changes needed to accommodate sustained business performance and allow for continuous optimization.
Having an underlying Service Portfolio that allows the selection and combination of those services in nex contexts into Business Processes is one of the key factors in achieving and maintaining sustainable Business Performance.
I will discuss other factors in my next entries.
A lot of market buzz is going on around BPM (business process management). This is a resurrection of one of the buzzwords of the earlier decade. However this time, it is BPM powered by SOA. Service-orientation is the underlying factor that will make a big difference in BPM. Rather than Business Process Management being solely about the management and optimization or re-engineering of business processes in traditional workflow contexts, with SOA, the activities the underlying orchestrated process (composite services) can be flexibly changed within a shorter duration, provided governance is fine with the change, and thus enable a greater degree of business agility.
As the need for IT to absorb variations increases, with the demand for greater business flexibility,we are confronted with some basic questions: how do we design simple for today, to get the current job done, but not "box " ourselves in a corner so we can support the required flexibility ?I think one major answer to this problem is Variation-oriented Analysis and Design (VOAD). VOAD consists of three main types or axes of variation: structural (type|data), process and policy/rule variations. Structural variations are often based on the identification of Types: Customer Type has variations like Gold Customer, PLatinum Customer and Normal Customer. Often this relates to variations in the structure of the class, or of attributes and data associated with the entity (looking from both OO and Data views).Process variations are when you recognize that a Gold Customer may start from a common base, but branch out into a different set of activities for registration or loan processing, for example. Policy or Rule Variations relate to the Rules and Policies (Rules about rules) that apply for each Type of Customer, for example.Clearly these three aspects of VOAD are related and complementary. It is often useful, in practice to distinguish and treat these three axes of variation.Once you analyze the variations along each axis of variation, you come up with a set of variation points: things that will tend to change or remain less stable. Deal with each variation point by applying a pattern. For example if a variation point for calculation of interest is required for various types of Customers, then a Strategy Pattern would be used to handle / instantiate that variation point.Note that variations tend to occur across all layers of an SOA..Not all that changes is a variation that is warranted to capture and model: only those that are architecturally significant will be worth your while to consider. How do you tell? An architecturally significant variation is one in which impacts the architectural /design decisions you will make and have a trickle down effect that will influence subsequent decisions in how you will build your architecture (e.g., SOA). The "domino" that will alter the course of other decisions is a significant or relevant variation.ANother FAQ is why focus on variations? Variations are more difficult to handle than commonalities. Previous literature focused more on identification of commonality which IMO has less of an architectural ripple effect than understanding, isolating and externalizing variations.
What do you think?[Read More]