Data Science, Machine Learning & API / SOA: Insights and Best Practices
Ali_Arsanjani 120000D8QB Tags:  soa manners externalization voad variations voa 2 Comments 6,931 Views
In an effort to be responsive to a number of requests for an update to my old manuscript for "Principles of Advanced Software Engineering: Variation-oriented Analysis, Design and Implementation", let me give you folks some background on the subject. [Background]Originally, I wrote a manuscript back in 1999-2000 and used it to teach my classes of Graduate students in computer science for my advanced software engineering class. Since then I and a number of my colleagues have written several articles on this subject and have extended it into various domains: Variation-oriented engineering, variation-oriented requirements analysis, VOAD for SOA Solutions in addition to the base variation-oriented analysis and design. This topic also relates to my work on patterns for stability and symmetry in software architecture which I was writing for the pattern languages of programming conferences. I also did a blog entry on VOA that describes the high level basics of this paradigm.[More Current] VOAD is an integral part of the SOMA method for building end to end enterprise and applications solutions based on SOA principles (whether or not you have a full SOA implementation in mind. VOAD intersects with another very important notion that I have been researching and applying on projects, namely, context-aware services (CAS). Manners externalize semantics for on-demand composition of context-aware services.Manners define the context-aware behavior that is needed in order to externalize component manners to achieve greater maintainability and reuse. This article shows how you apply VOAD in order externalize the changing aspects of the software, so you can decrease maintenance costs, enhance reuse and actually increase flexibility in design and implementation of your software solution. This is a rather popular article and has been cited in a number of other publications . I think this is because we show the practical application of VOA to create a dynamically reconfigurable architectural style. Also, I believe this element is one of the characteristics that will be needed in more mature SOA Solutions. So as organization's adoption increases, they will need more mature ways of doing SOA, and VOAD is a key to achieving that.Empowering the business analyst for on demand computing explores Grammar-oriented object design (GOOD) which is the way in which we apply VOAD for externalizing manners. This article describes a project we did in the public sector for harmonization of multiple redundant business processes across the world. This is immediately reminiscent of the value that VOA brings: one of which is to consolidate business processes and optimize them by eliminating redundancy.I suspect that is why I am getting these emails on putting more out there on VOAD and to update my older work to reflect the impact and opportunities for SOA. I get the message :-) and will do so.
The IBM Service-oriented Modeling and Architecture(TM) (SOMA) is IBM’s end to end software development and integration method (aka, process life-cycle) based on service-oriented software engineering principles to produce enterprise and individual project-level SOA or non-SOA solutions. This method includes prescriptive guidance applicable during the phases and iterations required in a software development process life-cycle. This guidance includes the activities and tasks to be performed by various designated roles within the life-cycle requiring input work products and producing or updating output work products and deliverables at specified junctures within the life-cycle. The tasks leverage a set of techniques (“capability patterns”) that prescribe and promote the use of best-practices included in the method, in conducting and accomplishing the task and to instantiate the related work products and deliverables.
In the world of SOA, the economics of funding determines what services actually get funded to be developed. This is why we need a set of criteria by which we can assess which candidate services are a priority for inclusion in your next budget cycle. We call these criteria or gating factors, the Service Litmus Tests (SLTs). SLTs are included in SOMA , the de facto end to end SOA Development Method .
Not all candidate services should be implemented as services; we may not have enough budget for that. But here this: we STILL need the functionality of the services that fail to pass the SLTs. Therefore those services still need to be implemented as part of some service component or realized by an application.
Economics is hand in glove with governance and of course SOA Gov includes the issues of economics, funding and budgets and ownership.
Ali_Arsanjani 120000D8QB Tags:  service-orientedcomputing maturity pervasive services 5,969 Views
SOA continues to evolve through economic downturns and bends in the road. SOA has evolved into a set of underlying principles, concepts, technologies, paradigm and patterns of designing and building software; all predicated on a set of loosely coupled services that collectively support an organization's business processes and goals.These services provides the enterprise with an adaptive or agile capability to morph and transform itself in the face of environmental change.
As we look at the SOA Maturity Model we have called SIMM (Service Integration Maturity Model), we see a clear evolution path:
1. Simple Services (level 4.0 to 4.9) from 2005 -20072. Composite Services (level 5.0 to 5.9) from 2007 - 2009
now we see a lot of interest in paradigms and technologies that are BASED on Service principles, namely
3. SIMM level 6, Virtualized Services, in the form of Software-as-a-service (Saas) and Cloud Computing.
Services have become Pervasive, they can be found in many of the recent initiatives including Mashups, BPM, Infrastructure, Enterprise Architecture, Governance, etc.
The notion of an SOA is finding new applications in new areas and continues to morph and integrate into new things, ingesting new use-cases but retaining the base principles of service-oriented computing.
As the eco-system of business evolves into a more opportunistic convergence of business partners into an eco-system of value, similar to the notion of a value chain, but a more loosely coupled exchange of goods and services, the need is increasingly being felt for an upfront design of an architecture structure to enable smooth and measurable business operations.
Eco-system architecture defines the foundation of interaction between a set of semi-autonomous business entities that interact with each other using a set of configurable policies that define the valid business interactions among potential and prospective participants, such as business partners: providers, consumers and brokers.
Eco-system architecture extends Enterprise Architecture in a similar way that EA extends the notion of solution architecture. EA is analogous to "city planning" whereas ECSA is analogous to planning how the economy of the whole nation is structured through a set of rules, policies, standards and structures .
Greetings Dear Readers.
I would like to pose a question, to which I would like to invite your responses or further elaborations. Then we will discuss this in greater depth, as I begin to discuss my own views on this.
In your opinion how does a company achieve and maintain sustainable business performance?
What are the factors impeding the achievement of desired business performance?
SOA enables business agility. It enables flexibility of IT systems that support the business architecture. As the business changes, the IT systems required to support the business in a volatile environment of competitive engagement, are less prone to change.
The history of software engineering started with the separation of functionality or processing and data. The data of the processes that manipulate the data were traditionally separate. There came a time when as the pendulum swung emphasis was placed more on the data in one era ended and the pendulum would swing and emphasis would be placed more on process. Eventually object oriented programming, which evolved into object-oriented design and object oriented analysis, played a key role in the unification of process and data. David Parnas was the first to suggest the notion of information hiding. The notion of information hiding was that access to information or access to data was strictly done through the functionality provided by an object. Those that object encapsulated its data and protected it from direct manipulation. It offered a set of operations that you could perform on the data, but you could only invoke those operations do not access the data directly.
Objects often reflected real-world objects. The identity of the real world object was used as an abstraction and implemented in a software system as a software object. Identity often corresponded to a real-world entity although helper objects evolve from the IT constructs necessary to implement the real-world construct in a computer system.
One of the most important principles of object oriented programming was based on a separation of concerns not only in the domain of abstraction but also in terms of separation of interface from implementation. Not only did we separate data from the operations that manipulated the data all in one object, but we did so one step further by separating the interface of the behavior from the actual implementation of that behavior. The behavior or rather its implementation, actually change the state of the object or as it were manipulated the data directly. However, there must be several ways in which we can implement this data manipulation. Even though there may be several ways in which we can implement this manipulation, there is typically one way to represent the interface or the externally visible signature that would be used by a consumer to request a change to be made to the data that the object owned.
This principle is called programming to interfaces rather than implementation.
These interfaces coupled with the notion of composing a set of similar objects that often had interdependencies among them into the next level of encapsulation which was called the component lead to a whole new era of -based development. It exported interface and could contain multiple objects all of which would typically be expected to be related to one another in some logically cohesive fashion. These objects were expected to be highly collaborative with one another and thus made sense to be called located within the packaging of a single component. This allowed not only manageable functionality but also decrease the risk that nonfunctional requirements would be violated.
The promise of object orientation
Architecture, software architecture specifically, used to be only defined in terms of components, connectors and constraints per Shaw and Garlan.
However, it is increasingly evident to those of us who implement architectures on a daily basis for clients and internally within our own companies
that architecture requires the addition of composition, context and containers to make it operational.
Components -- what are the structural building blocks of a solution, or a style of architecture, of the elements that can be combined to produce a larger structure
§Composition -- How to compose or what are the valid compositions of components
§Connectors -- The components need to connect with one another, whether statically (as in an Entity-relationship kind of relation) or dynamically, as in a composition in an SOA, where you may have orchestration or choreography.
§Constraints -- The constraints on the connectors and/or components that provide rules of engagement of what is permissible and what is not.
§Context -- the context of invocation of a component is critical to the designation of how that component will behave
§Containers -- the components must live in some runtime container that will provide uniform life-cycle management capabilities for them
Towards a more precise definition of Business Agility: The Building Blocks of Business Agility (B3A)
Ali_Arsanjani 120000D8QB Tags:  building_blocks_of_busine... agility business_value business b3a 5,321 Views
Business Agility is an often discussed as a key desirable attribute. One of the ways of achieving Business Agility is through the portfolio management of a set of business capabilities and services , rather than application portfolio management which tends to intertwine the services and higher order business capabilities into an application -- not a robust and flexible way of enabling business agility and optimization.
The main building blocks that provide a platform for business agility are outlined below. I have elaborated the last one more than the others in this post, and will elaborate further in future posts.
Laws governing eco-system as a whole (e.g., to disallow financial market meltdown in the wake of blind rule based automated short selling in e.g., financial markets)
The ecosystem includes policies, rules and laws governing Business providers and Business Consumers, and Business Brokers.
environment, market, legislation and ecosystem within which the business is
operating and evolving, forcing to change and vary based on forces within the
Business Context. The Business Context is a
There are many changes that are constantly occurring within the business context. Only some are business significant or should be “bubbled up” or surfaced at an executive and business level, especially when Business Sensors detect a certain threshold above which apparently ordinary events become important enough and achieve a tipping point beyond which the business sensors should indicate that a potentially impactful variation has occurred.
Business Entities model and reflect the key business domain elements of the Business Eco-system that under Business State changes.
Many EA (Enterprise Architecture) practitioners are …exploring the links between desired business outcomes and architectural decisions.
A tipping point is reached when IT begins to understand that business executives are not primarily looking for products and services but rather looking for business outcomes including increased output, higher quality, lower costs, increased revenue and increased market share.
To effect the transition to become more business-driven and engaged with business leaders, we should focus EA efforts on business outcomes.
Instrumentation and mechanisms to monitor, track, help modulate and govern business change as key performance indicators track the events and occurrences within the Business ecosystem Context.
Typically, the board and top executives of a company state the vision and strategy for the company. It is critical for implementation of business performance solution in the company that the senior business leadership translates and decomposes the high level vision and strategy statements of the executive leadership to operational and actionable goals associating KPIs to the actionable sub-goals on the way.
To achieve such end-to-end monitoring, metrics and sensors must be injected in both planning processes and delivery processes .
Essential to this process is the Business Sensors that detect Business significant Events and pass them on to the Business Policies and Rules that would respond to them.
From a planning perspective, enterprise governance needs to ensure that the right changes are initiated for the right reasons at the right time. The underlying premise driving towards business agility is that such agility delivers superior business value. But what if haste to achieve agility results in low quality? Or what if speed of change is unsustainable from a business operational perspective, thereby leading to deteriorating efficiency? These are just two examples of the fundamental challenge that doing the wrong things in the wrong way very fast simply means ruining your own business very fast. There are two fundamental premises for agile change to be valuable over time:
For agile change to be sustainable the enterprise needs to carefully plan and maintain an appropriate balance between effectiveness and efficiency. Change in the large is based on continuous business re-engineering towards strategic objectives (effectiveness). Yet while on that strategic journey an enterprise needs to apply change in the small to continuously adjust and optimize the current state and ultimately maintain business integrity and performance (efficiency).
There are two levels of monitoring – there is the monitoring of the acting on plans and there is the monitoring of operations.
From a delivery perspective, the set of solutions designed and implemented to achieve the business goals should be designed for monitoring. Monitoring of business results is a key to business performance. Monitoring of business results enables sensing of business significant events. Ability to act upon to respond to the business significant events resulting from the business activity monitors is essential to improve the business performance in order to meet the business goals.
Therefore, business goals, solutions adequately instrumented for monitoring and alerts, monitoring of business results, sensing of business significant events, responding to these business events to take corrective or predictive actions are key for a business to be in a path of success and growth that can be adjusted and tuned based on actual results of the business.
What is Cloud Computing? Similar to the Initial era in which SOA was trying to be defined, we are now in the era of more definitions. That is not in itself a negative thing, but a necessary part of the formation of new technology as they become mainstream.
Cloud computing is essentially an extension of SOA and service oriented computing. It provides concentrated infrastructure, platform and data center like capabilities offered at a set of services over the Internet. The cloud has several unique characteristics.
The cloud itself needs to be viewed from both a consumer and a provider perspective.
from the perspective of the consumer they are looking for a service-based delivery model over the web in which a highly reliable efficient self optimizing and feed your head up infrastructure capability are made available to them.
From a Provider perspective, the cloud involves two key aspects of self optimization and virtualization using a service-based delivery model.
The service-based delivery model can operate under one of the following kinds of services: platform as a service, software as a service an infrastructure as a service.
in order to connect the consumer with the provider, the provider must publish in some metadata standard information interchange format the quality of service that it can provide and the services that it can provide, so the service consumer can select among various provider of various services they provide and the various qualities of service that are offered for each type of service.
Could computing is fundamentally a service delivery model. It is instantiated by an architecture that provides the services and is consumed by a Service Consumer's own internal systems.
cloud computing is a
service delivery model
in which a virtualized set of
are offered by Cloud Service Providers with the choice of quality of service.
Service Consumers then have a choice
as to what provider to select
Further more, the model should support a seamless portable view of service such that services
from multiple providers can be choreographed by a consumer or broker without having to
know about the inner workings of the underlying cloud infrastructures.
What do you think?
Ali_Arsanjani 120000D8QB 5,215 Views
Amidst the current economic downturn, and in the mist of the survival instinct that is perculating around the world, it is tough to sit back and observe trends.But one unmistakable trend is that countries are shifting from an industry-focused economy (e.g., manufacturing, see the collapse of manufacturing toward a service economy. The impetus for this gradual change include globalization, technological change,and an overwhelming demand for (cheaper and cheaper) services. In observing this shift, it becomes increasingly clear that services and the service economy play an important role in today's and tomorrow's businesses. As a reflection of this trend, a realization or crystalization of the trend, service ecosystems have emerged: including GrandCentral, eBay, Google Base, Amazon.com, SalesForce.com, and SAP Business by Design to name a few. Such marketplaces enable trading products and also services between various legal entities and with consumers (sometimes referred to as the anonymous legal entity). One major challenge for service ecosystems is the fact that services are not products or goods. This has the potential to change the game.
George Galambos and I gave a talk last week at the annual IBM Technical Leadership Exchange about a topic that we have been talking about for some time now: the service eco-system. If you want to build a service eco-system you may need to look at some patterns that help you build your service eco-system .
As we move towards greater maturity in the adoption of SOA,there are a number of key challenges that need to be overcome. I like to call them the Grand Challenges of SOA. The first 9 are:
1. Business Case for SOA
2. SOA Maturity and Roadmap: What do I do next?
3. Service-oriented Modeling and Architecture end-to-end
4. Industry Specific SOA
5. Building composite applications
6. Monitoring and managing across the eco-system
7. Governing the SOA eco-system
8. Eco-system flexibility with declarative policies, service management and externalized functionality.
9. Service proliferation challenges quality of service
What do you think?
[b]S[/b]ervice. Refers to how we use a service interface as a contract to decouple or loosely-couple a service provider from their prospective service consumers. Providers publish services, prefereably and increasingly in registries or service repositories (where they can be better governed). Consumers are looking for capabilities / functionality along with a set of non-functional requirements or service level agreements (SLA) that they want to be able to declaratively specify.
Services allow a business to expose its main operations to a well defined set of partners, clients and world at large, without giving direct access to its underlying IT systems, butthrough the surrogate of a web service.
The structure of an SOA, introduces services at an architectural level (a layer dedicated to services in the architecture).In object-oriented programming we tried to separate interfaces from implementation. IN SOA, we have carried that programming practice up to the architectural level and now have a layer dedicated to enforce that programming best-practice at a design level.
[b]O[/b]riented. The orientation is not object-orientation or component-orientation, but rather, service-orientation. This orientation is a tendency towards using services above the other options; not to exclude those options. Thus, when we apply the service litmus tests to the portfolio of candidate services we will end up with a smaller set of services and a number of other capabilities that still need to be implemented using some technology: legacy, package or custom, even if it is not a web services implementation.
[b]A[/b]rchitecture. SOA is a style of architecture and relies on the sound principles of software architecture, including the fact that it is merely one style, to be combined with other styles to form hybrid solutions to handle complex projects and real-world situations. Booch talks about bringing the 'A' back into SOA. I would say that as we overcome the hype of the acronym, we turn our attention to each letter and take action on what it signifies; including the A part for Architecture.[Read More]
Ali_Arsanjani 120000D8QB 4,906 Views
To engage in an eco-system, you need to sense the environment, the eco-system. Business sensors gather business significant events and apply business policies to see what actions need to be taken as they sense a threat, an advantage, a change in the business climate, in the eco-system architecture.
Thus, to measure business performance and monitor business agility, you need what I am coining as "business sensors".
We have just published the SOA Manifesto . This document helps guide the values and principles of service-orientation and service-oriented architecture.
"Service orientation is a paradigm that frames what you do.A set of 6 values are then outlined, followed by a set of 14 principles that help guide the implementation of the values.
When faced with the forces outlines in the value statements, we provide a preference for one rather than the other, even
though both may be viable under certain circumstances.
I will post links to other authors of the manifesto soon.