 | Level: Intermediate Jeremy Caine (jeremy_caine@uk.ibm.com), Senior IT Architect,
IBM
Joe Hardman (joe_hardman@uk.ibm.com), Senior IT Architect,
IBM
19 Apr 2007 Service-Oriented Architecture (SOA) is at the heart of many
business-transformation efforts. Many enterprises approach SOA transformation
incrementally, using their valuable legacy IT systems to participate as service
providers. The solution architect's challenge is not only to deliver the SOA
infrastructure as a means to aid transformation, but also to ensure that
enterprise-wide business operations remain robust and compliant. Your enterprise
must develop an enterprise information-management strategy that can be part of the
SOA and maintain overall data and content consistency across all business
operations. Discover the challenges of such transformations, and review some design
strategies to consider.
Introduction
If your business is like most, your business strategy and core-competency
analysis have identified where you need to focus on business re-engineering and IT
modernisation. A typical outcome (at least for some of an enterprise's core
aspects) is the transformation to a service-oriented business and its dependency
on an SOA. Business operations are designed around a set of services that require
current IT systems to be re-engineered and integrated as service providers. The
transformation may also introduce new service providers. The goal is to be able to
build new composite applications, such as internal workplace portal applications
or Internet commerce applications, in an efficient way to address specific
business needs.
As part of this challenging transformation, enterprises generally engage in a
number of significant business and IT programmes. The primary transformation
deliverables include:
- Alignment of the enterprise architecture around a business service model.
- Introduction of a service-based IT infrastructure that can realise the
relevant parts of the business service model.
- Transformation and extension of, or applying a wrapper to, legacy IT systems
so that they can be integrated into this service-based IT infrastructure.
- A robust design and development environment and delivery processes that enable
efficient and flexible implementation of supporting IT applications.
These primary deliverables of the transformation create a secondary,
consequential set of challenges to implement. These ensure that an enterprise
delivers on its transformation intent and can monitor and adapt to change to
achieve flexibility. These transformation challenges typically aim to adhere to
the following principles:
- The enterprise must continue to provide an overall robust and compliant set of
business operations.
- A governance process and design authority is needed to ensure that the overall
business service model and IT system implementations adhere to the overarching
transformation principles.
- The enterprise must achieve an integrated view of business and IT service
management so that measurement and responsive change can be delivered.
Some enterprises are fortunate to be able to make the business transformation to
SOA across the entire organisation. This approach presents a more challenging
implementation for the primary transformation deliverables, but consequentially
provides an easier environment for growing and developing the secondary
transformation deliverables. This type of scenario usually occurs when C-level
executives who want to drive business and IT change across the enterprise provide
proactive sponsorship to aid their strategic agendas.
However, most enterprise transformations are incremental -- significant and
strategic, but nonetheless incremental. This typically means that legacy IT
systems are re-engineered so they can participate in support of new business
operations -- for example, combine with other IT systems to facilitate
straight-through processing of customer orders. These IT systems also continue to
perform all, or least a large majority, of their current capabilities. Business
operations outside the scope of the SOA transformation still continue.
In this transformation scenario, it's imperative that you consider careful
implementation design for the SOA IT infrastructure. Data processing within this
set of IT systems impacts and is impacted by data processing outside of that
scope. The overall data-processing environment must continue to be robust and
compliant for overall business operations.
The example in Figure 1 shows that three legacy systems have been engineered to
be service providers.
Figure 1. Legacy IT in new SOA
landscape
In Figure 1, the business-transformation programme has built an SOA
infrastructure by which business services are published and can be consumed by new
applications. This infrastructure typically includes middleware to enable
multiple-protocol connections between software, asynchronous and synchronous
message-flow management, message translation, rule execution, and process
choreography. Composite applications have been built that use these services, and
even legacy systems themselves have been enhanced by taking advantage of invoking
new services.
Process and information
architecture in an SOA
To implement an SOA infrastructure, the key asset that's produced, maintained,
and enhanced is the business service model. Well-understood techniques,
such as IBM® Service Oriented Modeling and Architecture (see
Resources for more information), are used to develop that
model. In a typical enterprise, this includes a mixture of top-down analyses,
bottom-up analyses, and goal-service modeling. The transformation's business
objective results in a process and information architecture that's used to drive
implementations of new business operations and supporting IT applications:
-
Process architecture is a functional hierarchy of actions that can be
combined to form a set of business processes underlying enterprise operations.
It's responsible for flow control of humans to systems and system to systems.
For example, it might include the actions that need to happen for a customer to
place an order for a product and be put into a satisfied state waiting for the
delivery of that product.
-
Information architecture is a set of related data and content that's
accessed, manipulated, and stored to provide state and meaning to the enterprise
operations. It performs the information management of data and content, whether
in the context of a simple access event or process flow. An example is the data
and content that's consumed and generated during a product-development life
cycle (concept to manufacturing, and so on).
The process and information architecture viewpoints are often developed together
and demonstrated through such things as a process-data (CRUD) matrix.
Collectively, they define a number of services that compose the service
architecture. A service is a set of operations called where message data is passed
in. The service implementation performs some processing that manipulates some data
or content, usually with persistent storage or access, and returns a result.
Services can be used synchronously and asynchronously, and invocation is generally
brokered by an integration infrastructure. In a completely decoupled system, some
services are responsible for causing changes to process state (for example,
information presented as requested, new address for customer X has been
validated). Other services are responsible for causing changes to the information
state, ensuring data and content have been manipulated into a desired state that
future service events can rely on (for example, a new shipping address is in
effect for customer X).
 |
Legacy data-processing
challenges
Legacy IT systems will undoubtedly be part of your enterprise's service-provider
implementation and be integrated into the SOA infrastructure. Legacy includes not
only traditional mainframe processing, but also enterprise resource planning (ERP)
applications, internal and external Web applications, workflow applications,
scanning and printing systems, and so on. A number of things can happen to enable
these legacy systems to be used and combined into a service-provider
implementation:
- The processing function is exposed and made accessible so that it can be
combined into a service-provider implementation. This frequently means exposing
a transaction via a new protocol and invocation mechanism, such as SOAP wrappers
on CICS® transactions.
- Functions may need re-engineering to maintain processing integrity when
exposing the transaction. The transaction might be something existing or created
(facade) to call one or more other existing transactions. For these first two
bullets, a service-granularity strategy should influence the design of the
invocation call to these exposed services.
- Certain processing functions may need to be "switched" off because data
processing is now carried out by the new functions being invoked through the
service-provider implementation.
- Existing access rights as well as security and audit requirements must be
enforced for the new access mechanisms.
- Quality of Service (QoS) criteria for existing systems may need to be extended
or supplemented. For example, off-line batch-processing windows may need to be
covered by stand-in processing solutions.
- SOA may introduce a new and unforeseen load on existing systems, which
negatively impacts existing processes.
When the legacy IT systems are exposed in an effort to participate in the SOA,
certain functions that support business operations outside the transformation's
scope continue data processing. This typically requires understanding and dealing
with data currency and synchronisation issues.
Users continue to invoke the legacy IT systems using the original application
means, typically through a user interface (such as a mainframe dumb terminal or a
packaged application's UI). The SOA depends on a set of data schemas that are used
to describe the messages that flow around the SOA ecosystem. These data schemas
are realised as physical data messages that are integrated within a collaborating
set of service operations; this can include runtime data translation and semantic
mapping, or augmentation of message data. This data is then used and processed by
the underlying legacy IT system.
The set of processes that can be executed with the SOA relies on a collective set
of data and content that needs to be in particular states. The parallel processing
via SOA functions and legacy IT systems can cause instability in that state model.
In the example in Figure 1, the transformation has enabled a
new Internet application that has straight-through processing capabilities to set
up and administer insurance policies for customers. The new applications can
trigger contact with the customer and offer a renewal deal when an insurance
policy is coming close to renewal. However, some existing business operations
outside of the SOA scope could cause problems. Customers have the means to notify
the company by any number of routes that they have moved residence. Even though
the policy was first purchased through the new Internet application, a customer
may not use that SOA application to inform the company of an address change,
perhaps sending a letter instead. A back-office process causes the customer's new
address to be applied only to the customer relationship management (CRM) system.
Overnight the existing CRM system's batch processing updates the Insurance
Contract Administration system with customer data updates. Because the new
SOA-based systems are designed around near-real-time transactions, data
synchronisation timing issues now result. During that time, the new application
could try to contact the customer about renewals at an old address, when in fact a
responsive and context-aware SOA application should be contacting that person at
the new address about cancelling the old policy and setting up a new one.
|
Hang on a minute,
service provider
The SOA architect's role should theoretically be concerned with the design and
implementation of a service-enabling infrastructure. SOA modeling techniques guide
you to decide the services your process and information architecture will consume,
contract with a service provider, and build applications that invoke service
operations as part of process execution and data processing. The service consumer
doesn't care how the service provider implements the service as long as it states
it will process and manage data according to an agreed set of contracts (service
operations and their associated QoS).
In certain scenarios this model may work, especially if you're sourcing
service-provider implementations external to the enterprise. This article focuses
on specific but common scenarios being exercised by many enterprises adopting SOA
today.
The reality is that an overall solution architect in an SOA transformation has
many responsibilities, especially in an enterprise that's making incremental
transformations. Those responsibilities are carried out within the bounds of the
enterprise governance process (see the article
"A case for SOA governance"
[developerWorks, August 2005] for more information).
First, the solution architect is concerned with designing and implementing the
SOA infrastructure; this includes identifying the legacy IT systems that will act
as software systems that form part of a service-provider implementation. Second,
the solution architect provides the development processes and environment for
assembling service-consuming applications. This includes design patterns and
standards as well as tooling environments. But third, the solution architect must
be cognisant of the overarching business and IT architecture responsibilities. The
business architecture will state what is reasonable for the business strategy with
its sourced capabilities and contracts. The IT architecture will deliver IT
systems necessary to support the overall business operations. Within any
reasonably sized to large enterprise, this is usually a large portfolio of
tactical and strategic applications.
Design strategies to
consider
To build a robust information-management architecture that enables overall
enterprise compliance, you can consider a number of design strategies and
principles that address the issues.
Plan for scope changes in business
process analysis
During a transformation programme, a common scenario is that system analysis and
design discovers undesirable data-processing scenarios. Business operations that
were previously out of scope might need to be brought into scope so that they can
be modified to accommodate a new set of systems. The new SOA-based systems may
even need to provide internal applications for these changed business operations
to use, such as an extra processing step to update some customer details.
Publish and subscribe to legacy entity
changes
When agreed entity and attribute data is changed in a legacy IT system that's
participating as a service provider, extra processing needs to be developed to
inform the SOA. Ideally this should be real time, but more often than not it's in
the form of a scheduled batch activity. Applying instrumentation code to legacy IT
systems that detects when data records have changed is likely to be costly, if not
nearly impossible, in older and unstructured application code.
One thing your enterprise must do is assess the data owned by a legacy system and
the system's capability to notify interested parties of changes. The high
frequency of changes to data that's important to the SOA, combined with a legacy
system's inability to generate record-level event triggers (so a daily batch
extract might be best achievable), might lead to a decision to tackle
re-engineering or replacing the legacy system. Make sure the business services'
QoS is aligned with the planned engineering effort. Indicate how the QoS will
improve over time, with the planned increment engineering improvements, such as
moving from daily batch processing of key service data to real-time message-driven
processing.
Information service query facade
Operations of the information-management services should present simple actions
that can be applied to data or content. The query facade hides how the information
is accessed and manipulated, and across how many physical IT systems. Basically,
the query facade can implement two patterns to achieve data integration. Because
of the variety of scenarios and legacy systems within an enterprise, it's most
likely both will be used to implement the services:
-
Take the data to the query. Collate slowly changing information into a
holding data store (a warehouse, for example), and act upon the data there.
Extract, Transform, and Load (ETL) technologies can be used here.
-
Take the query to the data. For rapidly changing transactional data,
the action must go to the physical source to manipulate it. You can use
connector and adapter technology to make those accesses (for example read and
update).
Within the SOA, the actual query facade could be implemented using a variety of
patterns, such as federation and population (see Data Integration
application patterns in Resources for more information),
and technologies like Service Data Objects and Data Access Services (see
Resources). The overall design patterns (get more
information in
"Toward a pattern language for Service-Oriented Architecture and Integration, Part 1"
[developerWorks, July 2005]) for the various service implementations will
influence these decisions. But in this problem space, the implementation design
would need to be extended to make the service more intelligent. On the assumption
that solutions are in place (to agreed levels of QoS) for legacy systems to
publish entity data changes, the information service needs to subscribe to the
changes (which and how often) and apply the changes to respective underlying
physical systems, such as those that perform the data synchronisation.
This is the place in the architecture where you make master entity decisions.
Which legacy IT system should be regarded as the master and in which transaction
scenarios? This is where master data management technology can help your
implementation, but of course the options and decisions, balanced with the
integration capabilities of the legacy system, should be a significant part of the
architecture definition.
Business-rules placement
The transformation to SOA introduces a new IT infrastructure that includes
integration middleware and new composite applications, among other entities.
Business rules are no longer contained within single applications. State is
effectively managed through the execution of business rules. As part of developing
the information-management design, the types and location of business-rules logic
must be understood.
The types of business rules within an SOA include:
- Referential-integrity rules
- Application-behaviour rules
- Cross-application integration rules
- Transaction-processing business-logic rules
- Data-entity processing rules, such as stored procedures
The strategy should be to reuse rather than reproduce business rules. With the
introduction of an SOA, you must carefully consider the execution of existing and
new business rules (to support robust execution of services). The rules logic
itself (wherever it's located in the context) might need to be exposed as a
service that can be called to maintain integrity for certain processing scenarios.
Potential
design-strategy impacts
The potential impacts to solutions that you must consider when reaching these
design decisions include:
-
Double-key processing. Efficiencies gained in one business area by
introduction of the SOA need to be balanced against extra data-entry tasks (and
therefore increased risk of data-entry errors). But it may be the only option
for coping with data-synchronisation timing scenarios. These scenarios are not
usually taken lightly. Your enterprise needs to establish a head count of
persons to perform event-driven double keying to avoid the time delay of
a nightly batch update. You'll counter this cost against developing an automated
interface in a system to assist in data synchronisation. This circular analysis
should assess whether the time required to implement automated interfaces
(especially testing) compromises the overall delivery time scales.
-
Unnecessary feedback loops. An update to a customer record in the CRM
system is followed by a manual or automated call to the SOA to inform other
applications of this out-of-bounds customer data change. To update the customer,
the SOA calls its information service, which in itself calls out to the original
CRM system and updates the customer data. The information-services architecture
needs to implement different styles of update customer depending on the
usage model.
-
Scope creep. When an SOA scope statement is defined, lack of knowledge
about legacy IT systems and what their process and information-management
capabilities are undoubtedly leads to change. The derivation of the service
model drives out that legacy analysis, only to reveal that a more substantial
SOA infrastructure implementation is required. Consider budget and planning for
phases of discovery before defining the overall scope of business services and
engineering programme.
|
Conclusion
This article has outlined some areas for you to focus on to deliver successful
business-transformation programmes incorporating SOA. The challenge of a business
transformation to an SOA is significant. Most enterprises approach transformation
incrementally by introducing an SOA infrastructure from which new composite
applications can be built. They use legacy IT systems to implement the service
providers.
For an enterprise to remain robust and compliant across all its operations, the
transformation programme must develop an enterprise information-management
solution, which includes — but is beyond the scope of — the SOA. This solution
should address the issues of consistency and synchronisation of data and content
across the legacy IT systems in the SOA so that information services can be safely
exposed for consumption.
The solution architect responsible for delivering the SOA transformation must
also balance the capabilities of the legacy IT systems against the needs of the
SOA in the context of the overall governed enterprise architecture and budget.
Technologies for data integration and master data management factor in an
implementation strategy for the information-management solution.
Acknowledgements
We would like to thank Rick Robinson, Patrick Dantressangle, Bob Lojek, and
Claudio Cozzi for their reviews and guidance.
Resources Learn
Get products and technologies
- Innovate your next
development project with
IBM trial software,
available for download or on DVD.
Discuss
About the authors | 
| | Jeremy Caine is a senior IT architect and technology strategy consultant with
IBM Global Business Services. He works on large business-transformation and
system-integration programmes, delivering complex IT systems, Web technology
solutions, and enterprise application integration strategies, primarily with
financial services and government clients. Jeremy is also an active member of many
worldwide IBM technical communities, providing thought leadership and supporting
the creation of offerings for areas, such as SOA. He is an experienced methods
leader and member of the IBM architectural teaching team. |
| 
| | Joe Hardman is a senior IT architect with IBM Global Business Services. He
specialises in SOA and Web application architecture and security. His recent
projects include the cahoot Internet bank and Polaris imarket, a Web
service-enabled e-marketplace for the UK commercial insurance industry. He is
currently helping a leading global oil company with its SOA programme. |
Rate this page
| |
