Visibility and traceability are two key imperatives for any successful model-driven development (MDD) initiative, especially in environments where multiple modeling tools are used in support of different roles and model types. Historically, the approaches to supporting these imperatives have been largely based on either an export/import concept or on the notion of a single global repository. However, both of these approaches have severe manageability issues in a dynamic distributed environment.
A similar challenge was faced in the early days of the internet and the Web pioneers chose a completely different approach to achieving visibility and traceability. The Web today is based on the notion of linking different sorts of pages through lightweight, standardized HTTP references and accepting the fact that such references might be broken from time to time as simply the price to be paid for avoiding the overhead of copying or the rigidity of a global repository.
In this article, we'll describe a linking approach to effectively interconnect modeling artifacts across different modeling domains, tools and repositories to provide for coherent visibility and traceability throughout an extended MDD environment.
With a proliferation of model-driven techniques and disciplines such as service-oriented architecture (SOA), business process management (BPM), information management (IM) and enterprise architecture (EA) many organizations find themselves faced with the challenge of managing growing numbers of model artifacts across domains and roles. Model artifacts may have affinity relationships, direct dependencies, or may even be different representations or viewpoints of what is essentially the same system. Often model artifact references are resolved by copying or transforming the models themselves for consumption in a different tool or domain. This approach creates potentially massive amounts of rework and institutionalizes the need for constant synchronization across environments and tools. Even worse, a copying approach blurs the lines of ownership and does not respect the different lifecycles of various types of model artifacts.
As mentioned earlier, visibility and traceability are two key parameters for any successful MDD initiative, but especially in environments where multiple modeling tools are used in support of different roles and model types. One way of creating visibility and traceability is by having all modeling tools use a single global repository. However, in many cases that is simply not practical either due to the tools involved, or the infrastructure mix, or simply because different roles need different user experiences leading ultimately to different model management requirements.
When collaborating across modeling domains, the typical approach has historically been to exchange artifacts by copying, in many cases even using transformations when doing so. There are several problems with that approach, including:
- Point to point proprietary integration.
- Loss of information through transformation or conceptual mismatch.
- No visibility across domain boundaries.
- Change management becomes complicated or impossible because it's not clear who owns the authoritative truth.
- No support for artifact lifecycle management.
Emerging industry standards such as BPMN2.0 and SoaML are incorporating quasi-normative guidelines that reduce the need for transformations and proprietary integration. Yet standard formats are no help for the manageability issues -- the only way of truly addressing those issues is to stop copying artifacts. The root of the problem is that when copying an artifact, you suddenly have two physically distinct copies of the same thing. This is subtly, but importantly, different from cloning an artifact to something related but distinct. The following examples should help clarify the distinction:
- Copying: Exchanging a service model between a service modeling tool and a process modeling tool in order to leverage that service model for process orchestration.
- Cloning: Taking an enterprise architecture process template and cloning it to a process model that is part of a particular solution.
In the first case, with the copy approach, if you changed the service model in any of the tools, you would definitely want to synchronize that change to the other tool since both copies are in fact one and the same artifact.
In the second case, with the cloning approach, even though the clone is initially bit by bit the same as the original (or possibly a transformation of it), the clone has its own distinct identity and lifecycle independent of the original. After all, even though you modify a process model for a particular solution, in most cases that does not mean that you want to change your enterprise standard. Having said that, it's still important to maintain a link from a clone to its origin in support of visibility, traceability and future collaboration. Refer to Continuous improvement with BPM and EA together (Claus T. Jensen, developerWorks 2010) for additional details on this collaboration pattern.
While this is just an example, it illustrates the fact that when crossing domain boundaries there is no good reason to do exchange by copy. Either a simple link, providing visibility and traceability is enough, or a clone that has its own unique identity and is linked back to the original is needed.
Whether a given situation can be handled by simple links, or whether it requires cloning, in both cases a standardized, tool neutral way of linking artifacts across modeling domains is required. Such standardized linking is one of the challenges being addressed in the Open Services for Lifecycle Collaboration (OSLC) initiative:
- Links must be transparent.
- Links must be bi-directionally visible.
- Links must be version sensitive.
- Links must be robust against change.
Linking provides for a much more seamless experience than copying and offers better visibility and traceability across role and tool boundaries. Process models can be linked directly to the service models that they orchestrate, with the dependencies visible to both the business analyst and the service architect. Services can be linked to the information artifacts that they depend on, and be visible to both the service architect and the data architect. Enterprise architecture targets can be linked to the solution models that they guide and govern, with the relationships visible to both enterprise architect and anyone working on the solution model in question. In short, we need to start using transparent links between artifacts wherever we can, as shown in Figure 1.
Figure 1. Internet style linking
Even in the case where a "copy" is truly needed, such as when seeding a BPM solution with an EA template, the original artifact should be cloned in a traceable and linked fashion, not exchanged by copy. This will provide an experience that is:
- People-centric, supporting collaboration across participants yet respecting domains of authority.
- Transparent, providing cross domain visibility via many-to-many artifact relationships.
- Managed, with tool assisted management of artifact links.
- Coordinated, with lightweight coordination focusing on synergies instead of attempting heavy complete synchronization.
Lately, the industry has put a lot of focus on executing agile change, yet agility at the cost of manageability is not very useful. In fact, one of the challenges a modern enterprise must meet is how to achieve continuous business improvement through collaborative and integrated planning and delivery processes across the enterprise. The meld of planning and delivery is exactly where the linking approach is strongest, as it allows us to respect lifecycles, domain boundaries and ownership, yet provides for holistic visibility and collaboration. See Leveraging SOA, BPM and EA for Strategic Business and IT Alignment (Claus T. Jensen et al, developerWorks 2008) for information on how to link BPM and EA for better business outcomes. Figure 2 shows an example of how BPM and EA can be linked.
Figure 2. Unleash BPM and EA synergies via flexible linking
This linkage of planning and delivery is important not only to IT, but to the line of business as well. Without proper integration of planning and delivery processes across the enterprise, business evolution becomes opaque and uncoordinated. And without rigor in managing architectural change across modeling domains, solutions may quickly become brittle.
Technologies to support a linking approach are rapidly evolving. Already many asset management repositories support any-to-any relationships between assets in the repository and provide basic elements of social collaboration. Moving forward, links need to be standardized and federated across repositories with the kinds of properties being addresses by OSLC. Indeed an amalgamation of basic linking with traditional elements of MDD source control, project management and tracking is a core element of the work ahead.
Model-driven development intrinsically requires collaboration and visibility across a multitude of different modeling and engineering domains. While each such domain historically has its own discipline and value proposition, a model-driven enterprise needs to ensure that each domain is synergistic with all the others. Leveraging such synergies at an enterprise level requires the establishment of appropriate collaboration and governance processes. From an organizational perspective, the enterprise needs to leverage the synergistic powers of robust architectural planning and agile business optimization. From a technological perspective, the enterprise needs to establish a platform that will enable the appropriate collaboration by creating visibility, traceability and integrity between targets and solutions across all roles and tools.
The classical exchange-by-copy approach to cross-domain collaboration does not scale well in a large model-driven enterprise. At best it results in a brittle and difficult to maintain model architecture, at worst it leads to misunderstandings and uncoordinated efforts that will significantly hurt an emerging modeling culture. Effective collaboration across modeling domains based on robust maintainable model architectures will be a key differentiator for successful enterprises in their drive toward continuous business performance and services optimization. To that end, such enterprises need to stop copying and start linking.
- Continuous improvement with BPM and EA together (Claus T. Jensen, developerWorks 2010): This article describes the principles for aligning and interconnecting BPM and EA from a business perspective. The primary audiences are leaders and architects who need to understand how to effectively combine BPM and EA as a key differentiator for successful enterprises in their drive toward continuous business improvement.
- Achieving business agility with BPM and SOA together – Smart work in the smart enterprise (Claus T. Jensen, Rob High, Jr, Steve Mills, 2009): This whitepaper describes the principles for the convergence of BPM and SOA. While BPM and SOA each have value on their own, IBM believes that they are naturally synergistic, and best when done together for business and IT agility, optimization and alignment. When done together, BPM provides the business context, understanding and metrics, and SOA provides a governed library of well-architected service and information building blocks. Both are, in fact, needed in order to dynamically optimize investments, drive operational excellence and manage business risk. Learn how to effectively combine BPM and SOA as a key differentiator for your enterprise in your journey to achieve business agility.
- BPM and SOA require robust and scalable information systems – Smart work in the smart enterprise (Claus T Jensen, Rob High, Jr., Steve Mills, 2009): This white paper describes the principles for the convergence of BPM and SOA from an information system perspective. When BPM and SOA are properly combined from an information systems perspective, then excellence in business execution brought by BPM relies heavily on the horizontal transaction processing and scaling brought by SOA. The net effect of exploiting this natural synergy is a trusted, consistent and managed network of interacting and interdependent people, processes, services and information sources. The primary audiences of this white paper are IT leaders and architects that need to understand how to effectively combine BPM and SOA in support of business integrity and operational excellence.
- Leveraging SOA, BPM and EA for Strategic Business and IT Alignment (Claus T. Jensen et al, developerWorks 2008): In today's enterprises, aligning business and IT to support business agility and transformation is essential. You can achieve this goal by applying SOA, BPM, and EA together in a synergistic fashion. This whitepaper describes key architecture and lifecycle principles to achieve that architectural convergence, and suggests adoption patterns based on the needs and maturity of an organization.
- Actionable Business Architecture (PDF) (Ray Harishankar, Kerrie Holley et al, 2010.): This paper poses and answers the question of what makes business architecture actionable, and discusses three specific perspectives: strategy and transformation (S&T), business process management (BPM) and service-oriented architecture (SOA).
- Open Services for Lifecycle Collaboration: Check out the OSLC community for more information on this initiative.
- Object Management Group Business Process Management Initiative: Get the BPMN 2.0 specification, articles, and more.
- Service oriented architecture Modeling Language (SoaML): Get the SoaML specification and other information.
- developerWorks BPM zone: Get the latest technical resources on IBM BPM solutions, including downloads, demos, articles, tutorials, events, webcasts, and more.