fiammante 100000A8UA 2,224 Views
My homepage is here. Has a little cooking, the van de Graaf I built, a little financial derivatives and futures math and some chemistry.
On the SOA front I had a very constructive discussion with Jérome Hannebelle from France Telecom/Orange (he wishes to be quoted) on a variability approach that differentiates the provider WSDLs from the consumer WSDLs. His interesting position is that to avoid the impact of version change the providers should expose more generic WSDLs with xsd:any for all the service message parameters branches that are subject ot release variations, however consumers should for the same service be provided with validation WSDLs that have explicit definition of the parameters for a given release. The provider then need at run time to identify the service request version an apply the appropriate routing and handling behind the service facade. This approach is a variation of the patterns I describe in my book where I already state that the umtimate provider's granularity and interfaces may be different from the consumer view. The implication is the dependency tree management in the registry that it implies to that there is explicit correlation between the various consumer validation WSDLs and the provider WSDL.
On the fun side Las Vegas is an interesting location, I flew a total of 15 minutes at the indoor skydiving tunnel. A safe way of experimenting skydiving feelings.
fiammante 100000A8UA 267 Views
In recent large projects I have been involved in defining an approach for capturing Non Functional Requirements, which are all quality aspects and constraints that will be required to satisfy in order to deliver the business goals, objectives or capabilities. These requirements are also necessary in a project scoping phase to evaluate project feasibility, costs and duration.
The “non functional requirements” term is often misunderstood by business users as they do not see why something would be “non-functional” in their requirements. This is why we are now rather using qualities (and constraints) to define these requirements.
In addition this is converging with the ISO 25010 that defines a model for Systems and software Quality Requirements and Evaluation (SQuaRE).
The divisions within the SQuaRE series are:
The ISO 25010 model defines the quality in use model (quality viewed from users of the system) and the product quality (intrinsic product viewpoint) models.
The “quality in use” model defines 14 characteristics and sub characteristics while the “product quality” model has 39 characteristics and sub characteristics shown in the table further below.
However not all qualities requirements will be the same for all of the architecture elements. For example the “Time behavior” quality, which could be a response time, can be different for an online account balance query versus a loan acceptance. On the other hand developing very detailed quality requirements is time and effort consuming, and there need to be a tradeoff between having enough requirements to size and cost, and spending too much time in trying to capture target measurements such as number of users, queries, locations etc.
Once defined quality requirement need to be defined with a precise measurement, such as Gigabytes per users, Screen size, response time in second, if necessary using examples from ISO/IEC 2502n - Quality Measurement Division. A verification method must be defined, so that users that see the resulting system can check that the requirement is delivered.
Some requirements are cumulative (storage, seats, physical space) so all of them need to be evaluated, for others the most constraining requirement is the one to capture (what is the most consuming user transaction on an ATM since only one at a time can be done).
Using functional decomposition to group requirements
In my projects we needed a way to group functions with similar quality requirements and we used functional decomposition to find such groupings.
For the purpose of the discussion to provide a public example on how we made this groupings I’ll use the BIAN service landscape see https://bian.org/servicelandscape/ (for telecommunication operators it could be the TAM https://www.tmforum.org/application-framework/ or a process classification such as APQC https://www.apqc.org/pcf ).
In the following picture a subset of the BIAN service landscape decomposition is on the left, and the ISO 25010 qualities are on the column headers.
As requirement groupings one can consider that all the quality requirements that relate to "collateral services" domain could be identical and captured in a single work product.
On the other hand it can make sense to have security requirements common to all of the enterprise.
Finally that does not prevent from having a specific set of requirements for a particular service, as an example the Fraud detection could be a highly restricted service and have its own requirements.
As a conclusion, the approach is to be as exhaustive as possible to avoid missing a cost factor, but also try to limit requirement capture effort by grouping as much as possible, and by only capturing requirements, that can be measured and verified.
Managing the complexity of business processes
Enterprises embarking on the business process management journey must ensure that they keep the gains that business processes provide by controlling the cost of their life cycle. Particularly, a process quality approach is essential to enable low-cost changes. This article should interest business architects, and IT managers and architects as it discusses an approach to controlling the development and maintenance efforts for business processes by limiting their complexity.
See the rest of the article on SOA Magazine .
Achieve Breakthrough Business Flexibility and Agility by Integrating SOA and BPM
Practical from start to finish, Dynamic SOA and BPM squarely addresses two of the most critical challenges today’s IT executives, architects, and
Navigating indirect documents relationships with Watson Explorer, Content Analytics, UIMA and recursive SQL
fiammante 100000A8UA 1,250 Views
In a project we use Watson Explorer and the Content Analytics Studio to create annotations such as Person, Location, Organisation, Account etc.
Watson Content Analytic is based on a Apache UIMA components in which the annotations attributes like name, birth date, address, car plate, account number etc. are called features.
Let's suppose that you have a first document with a person name and a car plate, in a second document a car plate and another person name, and finally a third document with that other person name and a location.
There is an indirect relationship between the first document and the third document, however using the usual queries that have indexed the features you cannot find the first and third document because the have no value in common. In addition as shown in the following picture the addresses do not have an extract match.
Luckily in the content analytic UIMA pipeline you can insert custom components written in Java that will add this indirect relationship capability that I shall describe further down in this entry.
Planning for the customization
To develop such custom Java annotator you can install the UIMA Eclipse tooling via the Eclipse update site: http://www.apache.org/dist/uima/eclipse-update-site/. This Eclipse plugin adds the "Add UIMA nature" menu option to Java projects so that the UIMA descriptor can be modifier with a specific UIMA editor.
The following pictures shows a UIMA custom component and some of its properties inserted after the parsing rules that in this article example are extracting the annotations such as name, location and organization.
Once the UIMA Java component is exported it can be added as a custom step in the pipeline displayed in the Watson Content Analytics Studio. In the following picture
Designing the resolution of the problem
In Watson Explorer the documents are crawled from wherever the sources have been configured (files, web etc), the extracted text then feeds the annotation pipeline, then indexing and annotation feature to facettes mapping occurs. You can add to the Watson Explorer GUI custom applications widgets to display additional aspects are needed as described in the following link Introduction to the IBM Watson Explorer Application Builder .
A custom application widget can display a tabular list of source documents and the documents they are related with directly or indirectly, with the links to these documents. It can include fields for further filtering.
But this widget requires this indirect relationship information with fuzzy relations to be displayed that has to be computed.
A rather easy way to navigate indirect relationships in any structured information is to use Recursive SQL (click to see Wikipedia article) where you look for a direct relationship and then recurse from the related documents to their own related documents.
In addition a commonly similarity matching between text is achieved by using trigram algorithm . Luckily some databases such as PostGreSQL implement this algorithm and even provide the "%" boolean operator. In a where clause text1 % text2 would return true if they are similar false otherwise. Click to see the PostGreSQL trigram article and click here to see a Java version of the algorithm if a database misses the trigram function it can be added using a User Defined Function (UDF).
Now we just have to implement a custom annotator that stores the documents, annotations and features in a database, and creates the appropriate views and queries serving as source for the Widget above.
Implementing the schema and the Java code
In the database schema we need the document,table with documents containing annotations in the annotation table, each annotation containing features in the feature table.
All annotations have a type e.g. Person, a beginning and an end location the position of the text in the document. All features have a type e.g. Name and a text value.
The following picture contains the resulting schema.
The view LINKEDDOCS returns the pairs of documents that have a similar feature value in commonn for the same feature names and annotation types.
CREATE VIEW LINKEDDOCS (PARENT, CHILD, VALUE) AS
The following SQL Query recurses the pairs of documents in the LINKEDDOCS view, limited to a depth of 10 to avoid infinite recursion.
Now we need to feed the database tables from the custom Java UIMA annotator that we add in the pipeline.
A java UIMA annotator extends the provided UIMA implementation e.g. public class Annotation2SQL extends JCasAnnotator_ImplBase.
There are 2 methods that need to be implemented "public void initialize(UimaContext aContext)" and "public void process(JCas aJCas)" .
The initialize method it called when the annotator is loaded and reads all the required parameters e.g. String dbName=aContext.getConfigParameterValue("JDBC","DBName"); and then that method also initialize the JDBC session.
The process method is called for each documents flowing into the pipeline, and a view of the document content with all previously created annotations is passed as the JCas parameter. There are several examples in the Apache UIMA tutorial or on the web. The first thing in the process is to locate the appropriate view as in Watson Content Analytics, the name of the associated CAS view is "lrw-view".
This is done by iterating the views "Iterator<JCas> viewI = aJCas.getViewIterator();" until the "lrw-view".is found if ( lrw_view.getViewName().equals("lrw-view")) break; .
We store the document information in the database using UIMA SourceDocumentInformation Java API (see link and source examples on the Web).
Then we need to iterate on the annotations and store then in the database using the following code
Finally for each annotation we iterate on the features, convert them to String and store them in the database
After the pipeline is processed all of the above tables have the information and the recursive query gives the indirect relationships.
InfoQ has just published a review of my book together with a Q&A here is the link of the interview and book exceprt:
InfoQ article link.
I wish you happy holiday with your family and the best for 2010.
I often see discussions comparing REST versus SOAP.
REST interactions are usually handled by the Web Server while by definition Web Services using SOAP are initial requester to ultimate provider interactions and the proxies and intermediate servers should be transparent. This is particularly true when using WS-Security with Signature and Encryption as only the destination application should decrypt and verify signature.
The following picture shows the SOAP and REST interaction on an OSI layer representation. See below this picture how to get both of both worlds using the WS-Transfer standard.
Now there is an easy way to get best of both worlds which is to use WS-Transfer standard where the payload is Resource Centric.
You can then get the benefit of the full protection and end to end encryption and signatures that do not stop at Web Server and get the flexibility of a Resource Centric approach.
WS-Transfer is recommended by several governments and organizations such as PEPPOL Pan European Procurement Online Protocol
Some of my client are worried about the differences between services, microservices and APIs, together with approaches like REST and they want some guidance on what to choose and how to weight the granularity of each option. A bit of recent history is necessary to understand the evolution from services to microservices.
in the 90's IT systems had mostly been developed as stove pipes and it was obvious that the Web evolution was requiring some integration, reuse and interoperability so that new processes and application could cut across the business domains. The Services Oriented Architecture was introduced at that time, looking at high value functional reuse.
From SOA to Microservices
SOA approach implies strong governance that implies control of the services catalog with service life cycle that are lengthy by nature. Because of the reusable services where at a high granularity with variability of interfaces. The granularity of these services can be measured using a functional decomposition where the top level is the Enterprise, then the business domains at level 1, then process groups at level 2, processes at level 3, services at level 4. At each level there are 7 to 10 elements which gives a level 4 catalog of potentially 1,500 to 10,000 services. Such decompositions are available from organizations such as APQC process classification Framework, banking industry association Service landscape, Telemanagement Forum Business Process Framework and their Standardized Interfaces and APIs, and identification methods such as IBM's SOMA.
But the new digital word requires faster responses and development, with agile methods and less constraining governance, somewhat contradictory with the enterprise wide service identification and control approach, hence the need for microservices. These microservices are much finer grained but they are not intended to have a complex life cycle with versions or improvements. If something different is need a new microservice is created.
My rules of thumb for microservices and associated components
Searching the Web the APIs appear to be a mix of services, microservices and other components of various technologies. A good example of that is the ProgrammableWeb API Directory that lists 12,000+ APIS of 26 technologies for 300+ categories/ functional domains.
The net is APIs include services or microservices, and that discussions on APIs have to go deeper in defining the approach that matches the enterprise need, particularly in terms of application delivery ecosystem and life cycle
REST aka "Representational State Transfer" is based on uniform resource identifiers (URI) that identify resources. Said otherwise resources are Business entities or "Data" with their access path. REST APIs are the combination of resources and verbs (GET, PUT, POST, DELETE, ...).
The granularity of a REST interaction is the depth of the URI path to access the target resource. In the figure below the top REST interaction has a depth of 2, while the bottom interaction has a depth of 4. A good practice is to keep the granularity under 3 which matches my microservice rule to focus on a data domain and its immediate dependencies.
I hope this entry helps my fellow architects and developer make their opinion about the topic and build identification and granularity evaluation approaches.
Seventh of Seven practices for Dynamic Process: Use Information & Event Centric Processes where appropriate
There are many processes that are information or business object centric. In such cases the entities have their own lifecycle and interact with the external world using services. Process analysts must however be careful on ensurign that they are now mixing business ownernship and separate the correctly the facettes that are owned by different owners in an organization. As an example the Telco standard information model clearly differentiates "the Customer Order" owned by customer facing organization from the "Service Order" and "Resource Orders" owned by other organizations in the enterprise.
Then the approach analyzes the state charts for each entity.
This approach can also be taken when different organization have to share a common entity such as a customs manifest declaration. The public entity and its public life cycle can be passed as an SCXML State Chart XML W3C SCXML Standard document. A reference implementation of SCXML available from Apache SCXML project.
The following picture describes such document bei exchanged between different organization and agencies, each with its own IT infrastructure. The only requirement is that each of those organization is able to interpret the SCXML standard to make out how the entity should be handled. It addition the document can carry the history of events that enables each stakeholder to reconstruct a monitoring view.
fiammante 100000A8UA Tags:  bpm business_process service variability polymorphism soa 3,030 Views
Variable Services or Service Polymorphism, in the context of object oriented programming, is the ability of one type, A, to appear as and be used like another type, B.
When applied to service invocation, we are talking about the ability to invoke one single service facade F, but actually invoke other services A, B, C etc, without the caller being aware.
This may look simple for Object Oriented Practictionners but Web Services Interoperability standard WS-I states that "Operation name overloading in a wsdl:portType is disallowed by the Profile".
The consequence is that we have to find service variability techniques that use Enterprise Service Bus mediations or other dynamic endpoint resolutions to solve the problem.
This enables the consumer to interact consistently with a generic service, and abstracts the many possible implementations, or specific services, which may ultimately handle the request.
In my book I have a full chapter on service variability and my colleague Scott Glen has written detailed articles in IBM DeveloperWorks address ways to implement service polymorphism.
Well here it comes again. Today I have been confronted to a project where the teams did not understand the need for variability and replaced the Characteristic Value variability pattern from the SID Telco model with static tags for each attributes.
At the end they make their interfaces signature rigid and will have to introduce new services for each different type of orders or products. This is typically what happend with Client-Server where changes on the server side propagated to the client side. We absolutely must avoid to use that Client-Server on Web Services approach for SOA as it ends up propagating and amplying the cost of changes to service consumers ,whether they are processes or other applications.
SOA needs to be about business or semantic loose coupling where the interfaces can absorb variations without having to change all of the consumers that use that specific interface.
The characteristic value pattern from the SID is as follows where the value point to a specification that defines the type, name and constraints. The information is structure in stable parts and variable parts with the variability of the information model enabled by this pattern.
An example of such variable message is also provided. The Characteristic values can be added to described very different aspects and do not change at all the structure of the messages even when adding new attributes to a product, service resource or a customer.
Have a nice day.
fiammante 100000A8UA 856 Views
I needed to check if a String contains another String with a fuzzy match. For this purpose I reuse the Levenshtein distance which is a string metric for measuring the difference between two sequences.
Since Leveinshtein distance measures the insertions, deletions or substitutions needed to get to the other string, the difference between the contained string length and the container string gives the number of deletions.
Substracting this number of deletions from the Leveinshtein distance between the two Strings gives the distance of the contained element from similar parts of the container String.
Here below is the Java code for the following result: String "street of the smoking dog" contains "the snoring dog" with a 15.0 % approximation
fiammante 100000A8UA 1,956 Views
I use the Business Process Modeling Notation (BPMN) categories as defined in BPMN 1.1 standard to categorize and modularize business processes.
There are three basic types of sub-models categories within an end-to-end BPMN model:
These models are use to create the end to end monitoring model by capturing events that surface from each of the smaller modules (the wagons).here is often a confusion between this monitoring model which higher management of the enterprise requires and the process automation which is provided from the next category of processes which are smaller modules.he monitoring model can take actions based on indicators it controls.
As I mention in my book we have too much considered SOA as client/server on Web Services. We really need to think variability as a way to enable reuse and avoid the propagation of provider changes to consumers and vice versa. In complement to the approach for variability I describe in my book looking at information variability, service variability and process variability here are two good articles on variability that an architect I work with in a large european bank just sent to me as we are working on these topics together for that bank.