SOA in action inside IBM, Part 2: SOA case studies

Export Validation Service and Common Customer Master System

Two SOA implementations illustrate the deployments of critical business services within IBM. In the first, an export control service allows organizations to ensure compliance with United States government export restrictions that define individuals, companies, or countries with whom U.S. companies are not allowed to conduct business. The second SOA service supports the management and availability of a complex set of customer information that is aggregated from a variety of sources.

Share:

Lance Walker (lancew@us.ibm.com), STSM, IBM CIO Office, IBM

Lance Walker is an IBM Distinguished Engineer in the IBM CIO office. He has application architecture responsibilities for several areas in IBM's overall enterprise architecture. At the time when this paper was written, he was the PDTL and lead architect for the corporate SOA+ Initiative (responsible to accelerate SOA for IBM Internal), and also the chair for the IBM internal SOA Guidance Council. Prior to this, he led other CIO application architecture initiatives, such as the Isolation Layer Framework. His previous experiences in enterprise common services, such as his role as the first lead architect for Web Identity (which creates common user IDs and passwords for IBM customers and business partners), has positioned him to understand customers' unique business needs and implementation constraints for SOA within IBM. His primary areas of expertise are SOA, enterprise architectures, Web Services adoption acceleration, and integration architectures. He holds Master's degrees in Computer Information Systems and Telecommunications from the University of Denver and mechanical engineering from the University of Colorado. He is also an IBM Certified Architect.



Akram Bou-Ghannam, Certified IT Architect, IBM Software Group, Application and Integration Middleware Software, IBM

Dr. Bou-Ghannam is a certified IT architect leading SOA enablement in EBI in the IBM enterprise for the past 3 years. Currently, he is driving the SOA transformation in EBI, focusing on creating reusable integration and business services, and on building SOA applications with reusable assets. For the CCMS case study described in this paper, he was the lead architect for the SOA transformation of CCMS. Prior to this activity, he spent 5 years in SWG product development working on WBI (WebSphere Business Integrator) products. He led the effort to create the IBM B2B connectivity architecture and was the architect and development lead on the WBC (WebSphere Business Connect) product. His areas of interest include SOA and intelligent agents technology. Dr. Bou-Ghannam has two patents issued and 16 patents pending. He received the IBM Seventh Level Invention Achievement Award, and has multiple internal and external publications to his name.



Luba Cherbakov (lubacher@us.ibm.com), IBM Distinguished Engineer, Member of the IBM Academy of Technology, IBM CIO Office, IBM

Luba Cherbakov, an IBM Distinguished Engineer, and Member of the IBM Academy of Technology, is a key technical leader in the IBM CIO Office. She is driving the CIO Situational Applications Environment Initiative that brings together Web 2.0 technologies, growing availability of services and enterprise data. She is a recognized expert in the architecture, design, and development of complex enterprise applications, using service-oriented and component-based methods. She is also an author, an advocate, and a contributor to the Service-Oriented Modeling and Architecture (SOMA) method, reference architecture assets, the Architectural Description Standard, and grid computing offerings. A member of IEEE Computer Society and ACM, Ms. Cherbakov has a Master's degree in computer science from George Washington University, with a major in software and systems and a minor in artificial intelligence and simulation.



31 October 2006

Also available in Chinese

Introduction

In Part 1 of this series we described six SOA business value propositions and the business drivers associated with each. We introduced seven IBM internal transformation initiatives that are enabled by SOA. These case studies represent a wide variety of business challenges that leverage SOA for the solution

In the first two case studies, we described Customer Order Analysis and Tracking System (COATS) and Microelectronics "factory in a box". In addition to the business context, we described challenges that the initiative had to overcome and architectural overview of the resulting solution, with its enabling technologies and tools. We also described tangible business results achieved by each, and best practices and lessons learned that were harvested and are being applied across IBM and with our clients.

This installation describes the the following case studies

  • Case Study 3: Export Validation Service (EVS) for regulatory compliance
  • Case Study 4: Central Customer Master System (CCMS)

EVS and CCMS represent key IBM internal solutions for compliance management and "information as a service" respectively. Given the increasing complexity of government regulatory demands, and the need to manage federated data sources to enable location transparency (please see the Resources section for more information), these two services are prominent examples of IBM's internal SOA direction.

EVS and CCMS represent key IBM internal solutions for compliance management and "information as a service" respectively. Given the increasing complexity of government regulatory demands, and the need to manage federated data sources to enable location transparency (please see the Resources section for more information), these two services are prominent examples of IBM's internal SOA direction.

Case Study 3: Export Validation Service (EVS) for regulatory compliance

Business context

United States export regulations define restrictions for what can be exported outside of its borders and to whom. These regulations apply to all situations involving any type of export, viewed in the broadest sense -- which includes the delivery of hardware, software, services, tools or technology from the U.S. to any other country. In addition to traditional physical shipments, the electronic transmission of software or technical information to entities (individuals, companies, organizations, governments) residing outside of the U.S. borders must also be controlled.

Certain products or technologies relating to national security, such as encryption, nuclear, chemical or biological warfare, or missiles, may not be sold or shipped to a prescribed list of countries. The U.S. government has published a list of entities, called Denied Parties List (DPL) with whom U.S. companies are not allowed to conduct business of any kind. It also publishes lists of embargoed or terrorist associated countries, which place additional restrictions for business engagements.

Challenges

Officially established policies to ensure compliance with government regulations are becoming increasingly important to be controlled at an enterprise-wide level to avoid costly penalties and other negative repercussions associated with violations. IBM’s Export Regulation Office (ERO) is responsible for how IBM must comply with the U.S. Government Export Regulation Procedures, and provides monthly updates of U.S. Government lists of people, companies, and countries with which the company cannot do business. IBM's online software sales and downloads (both free and priced software) require a more automated means for a real-time verification of the deliverable and to whom it was sent. This is necessary to insure the action did not violate U.S. restrictions.

Multiple applications have been deployed within IBM to support U.S. export regulations compliance. Several versions of export checking had been implemented by multiple areas of the business. They all needed to incorporate the ERO’s monthly updates of the control lists to avoid time-consuming manual tasks to recover from incorrectly initiated business transactions with entities on the prescribed lists. This proliferation of export control functions within the company resulted in unnecessary development and maintenance costs, in addition to the need for the ERO to work with several development teams to ensure compliance consistency.

IBM's Software Delivery and Fulfillment (SDF) organization had originally created functions to perform the various export checks for its own online sales and delivery capabilities. However, they needed a more efficient solution that would be easy to integrate for their own use, and it needed to be highly reusable to extend its capabilities to the rest of enterprise. SDF's business model allowed them to accept funds from potential service consuming organizations for new requirements, which facilitates a wider spread adoption of a service within a company.

The challenges that needed to be addressed by this solution are summarized below:

  • The U.S. Sarbanes-Oxley act has increased the awareness of the need for a new class of IBM internal services to support compliance for a broad set of government regulations.
  • Redundancy across applications and excessive development and maintenance costs,
  • Multiple applications had to be updated each month to incorporate the U.S. government's export control lists' to their localized copies.
  • If changes to the export control lists are not incorporated in a timely fashion, it may cause delays in IBM's Integrated Supply Chain processes.

SOA-based solution

Like several other early SOA business solutions, the basis of this service already existed in a Web application, which in this case, performed real-time checks for customers downloading software via a browser. In December 2003, SDF organization extended their browser-oriented foundation to create a SOAP-based Web service, which is now known as Export Validation Service (EVS). It allowed other requesters to call its customer analysis capabilities to comply with the U.S. government export regulations.

Based on the submitted customer demographic information such as name, address, location, IP address, and the IBM offering involved in a business transaction, this service currently supports export checking for several key areas:

  • User checking for 1) matching to the U.S. Government DPL, 2) screening to avoid the proliferation to specified countries of technologies that can be used for missiles, chemical and biological weapons, or nuclear purposes, and 3) matching to the U.S. Government's embargoed countries list (via address, email, or IP address)
  • Encryption products checking, to control the export of cryptographic products containing strong encryption
  • Unique export checks for Danish and Irish government requirements, which were needed to support IBM software fulfillment operations in these countries

The solution, depicted in Figure 1, includes decoupled enterprise data in the form of the ERO's updates of U.S. government Denied Parties List.

Figure 1. Export Validation Architecture Overview
EVA

In addition to the SOAP-based Export Web Service over HTTPS, the diagram illustrates the background processes to update the DPL -- manually access the ERO's official updated lists, and use the DPL Administration Tool to incorporate the latest updates into EVS's database. Also depicted are administrator functions to override false positives that occur when the "fuzzy" logic incorrectly identifies a customer on the prescribed lists (Override Administration Tool). This fuzzy analysis of the submitted information must be conducted to consider similar names and minor variations in spelling.

As with many business services, EVS is not a completely stand-alone function, it is integrated with a manual business process to provide additional value "behind" the service. There is the concept of the "Human Service Bus" which integrates human tasks in an organizational structure and is coordinated with a service implementation to assure compliance with a corporate standard (please see the Resources section about the article, Impact of service-oriented architecture on enterprise systems, organizational structures, and individuals), which in this case, also reflects a government regulation.

The development team originally used IBM WebSphere® Studio Application Developer™ , and then moved to IBM Rational® Application Developer™ as they updated their tools to the latest IBM products for application development. WebSphere Application Server Version V5.1.x, IBM DB2® Connect Enterprise Edition™, and IBM HTTP Server all run on AIX servers for their runtime environment.

EVS has already been enabled within several existing IBM business processes and will be used by new processes that require customer checking for export compliance. This includes transactions at the time of sale, (either on the Web or by phone), as well as a final check before the delivery of goods to customers.

Business results

U.S. Government export controls can now be more easily integrated in a real-time fashion into all Web and telephone interactions with customers. This service now allows centralization and consistency of all IBM export checking functions, and ensures that control list updates from the U.S. government are incorporated in a timely manner to eliminate the risk of government penalties.

EVS's service-based reusability decreases the cost associated with the development and maintenance of building redundant export control functions for multiple applications. In addition to cost, development schedules are reduced by providing the ability to consume an already existing service rather than building a new function.

EVS is defined in IBM's Enterprise Target architecture as a key SOA "Enterprise Component", which by definition designates it for reuse by all of IBM's business units. EVS is also being evaluated as a potential IBM commercial asset, whereby IBM can provide both the service and manual override process for a customer's business processes. Corporate and business unit architecture governance organizations work together, to enforce when applications must use these enterprise components, instead of building redundant functions.

Best practices used and lessons learned

Centralized funding of enterprise services is often a concern when addressing the requirements of multiple lines of businesses. Many enterprise services are centrally funded through their ownership by an organization that owns the mission of the service's business capabilities for the entire enterprise. EVS follows a different business model, whereby updates to the service for new requirements are individually funded by each service consumer group requesting the changes for their unique needs. This alternate model has been very successful to ensure that EVS is adequately funded to meet the challenges of new business directions.

As a result of this business model, however, incremental developments to meet the requirements of new consumers one at a time can sometimes leave fundamental improvements lagging. New consumers, who have their own unique requirements, are sometimes reluctant to fund general and underlying foundation elements of an enterprise-wide service. An expansion of EVS to a larger enterprise scope beyond software products requires updates for improved robustness to ensure that EVS can handle additional volumes. An example of this condition is older code based on Perl scripts, which must be converted to Java™, in addition to other performance improvements. General enterprise capabilities, both functional and non-functional, are being addressed incrementally by the EVS team when new clients have enterprise-like requirements, along with some guidance and funding infusion from the IBM Business Transformation CIO.

EVS demonstrates the need of consumer business owners to understand more than just their automated consumption of a particular Web service. A manual back-end process is currently required in EVS to follow-up on potential false-positive export check results. Many consumer development groups often focus exclusively on a service's interfaces, but also need to consider manual support functions, which can also enhance a service's usefulness.


Case Study 4: Common Customer Master System Single view of a customer within IBM

Business context

Depending on the channel used, such as call center, Internet, or business partner, IBM sometimes appears to customers as multiple companies operating independently. The customer’s experience, as well as customer organization and contact information, at times can be lacking in consistency and continuity. This has had a negative impact on customer satisfaction. To address this problem, IBM has deployed the Common Customer Master System (CCMS) as an SOA-based solution that enables a "One IBM" experience for its customers.

The key business objectives of CCMS are to:

  • Eliminate multiple instances of customer information that exists within IBM and the corresponding redundant maintenance expense by merging all IBM Customer data into a single system and associated repository. Information feeds from these other distributed customer repositories will be sunsetted and replaced by this single system.
  • Make customer information more meaningful and useful by incorporating organizational hierarchy into customer data. Organizational hierarchical structures provide business information on how different business entities relate to one another, their addresses and subsidiaries. These hierarchical structures must be updated monthly and be made available as IBM's trusted source for organizational hierarchical data.
  • Improve customer data quality.
  • Increase customer information utility by providing direct user accessibility via a graphical user interface (browser).
  • Provide service interfaces, which allow consuming applications to use this centralized system as IBM's single customer record creation and customer information source for all IBM customer information.

Challenges

The scope and complexity of the problem include:

  • The integration of customer data from different data sources -- including Dunn and Bradstreet (D&B), IBM's customer data stored in the Reference Data customer (RDc) hub (representing customer information from a variety of IBM customer databases), and CRM/Siebel (with three separate instances in North America, Europe, and Asia Pacific). This integration task also addresses the need to implement functions for data validation, the correction of duplicate records among the inputs from D&B, IBM CRM sources, and RDc.
  • Satisfy both user and application access capabilities with a singular set of SOA enabled capabilities
  • The wide range of requirements that can be generated with the need to support 119 different countries
  • The complexity of integrating multiple customer data models into a single logical model, which is consistent with IBM's standardized business data definitions to support the loading of D&B, CRM and RDc
  • Transformation of multiple, disparate, customer legacy data sources into a cohesive solution powered by SOA
  • The need to introduce new rules in a consistent manner that can be uniformly implemented, which include:
    • Business rules mapping for each data element from a data source's data models to the new centralized data model, including data quality and the list of acceptable value requirements
    • Transformation rules, by data element for each source, necessary to create and maintain data in CCMS
    • Data Aggregation rules, by data element for all data sources, also necessary to create and maintain data in CCMS

SOA-based solution

The CCMS team first identified and prioritized architecturally significant scenarios, which were used as the basis to define needed services. This analysis and design also involved the modeling of business processes created with WebSphere Business Integration Modeler, to identify and orchestrate the utilization of separately built services. The analysis was applied to each scenario to determine the optimum scope and granularity of the services, which involved refactoring to improve the decoupling of the services. Studies of CCMS legacy systems were completed to ascertain which existing software components could be "wrappered" as services, and which new services needed to be developed for functions not provided by these systems.

IBM Rational XDE was used to create UML use cases and object models for the IT artifacts, which were employed in the definition of the component model. Workflow based processes were designed with WebSphere Application Developer -- Integration Edition to assemble new services from a collection of existing services. The Rational XDE and Business Integration Modeler models were directly input into WebSphere Application Developer -- Integration Edition, which was used to associate additional artifacts (XML schema, Java code, services, and rules) with BPEL workflows, which were deployed onto WebSphere Business Integration Server Foundation.

The Core Transaction Update Service illustrated in Figure 2 is an example of a CCMS workflow based service, and is used by various update related processes, such as CRM update, RDc update, and D&B update to insure consistency, data quality, and to eliminate redundancy. This particular update service includes activities for address standardization, matching to determine whether the input record is unique or already exists, aggregation, and persistence. The aggregation activity implements data aggregation rules which, before CCMS data is updated, take in consideration the source of the data for each data group in the customer record to be updated. The aggregation rules indicate which data sources (per data group) to trust when updating CCMS data.

Figure 2 also illustrates the CRM update service, which implements a CRM update process and uses the Core Transaction Update Service.

Figure 2. CCMS architecture overview
CCMS

An example of the effort to convert existing assets into services is illustrated by the Address Standardization Service, the "Matching Service", and the Assign Key service. The services for address standardization and matching (checking if a record is unique or not) were created by wrapping functionality provided by an existing legacy Trillium engine. These services created by encapsulating Trillium can now be reused in other Customer Information business processes, and not just within the customer update process. Another step involved the creation of new services without a preexisting basis for reuse, such as the CCMS Persistence Service.

Business results

The SOA transformation of CCMS is a major step forward towards achieving the IBM business vision of an On-Demand enterprise. With SOA, CCMS is a flexible and adaptable solution, based on reuse. Enterprise business processes are reusing CCMS services instead of developing their own. The CCMS Update service, Search service, Address Standardization service, Match service are all good example of reusable services. Not only did CCMS provide reusable services, but now CCMS has the capability to orchestrate its services into enterprise processes. This flexibility powered by reuse provides great business value to IBM. The business value of this solution is evident, even though a measurable business value benefit in $$ has not yet been calculated.

Another evident business value for CCMS is the implementation of customer information as a service. Since customer information is essential for IBM's business, CCMS is one of the key IBM internal "information as a service". In the CCMS SOA solution information is packaged as a service to business processes, so that consistent, manageable information is made available to every process in a standardized way. In the pre-CCMS environment each business process created custom access to information which resulted in inconsistent views of customer data across processes, inconsistent application of rules, and multiple points of maintenance for the same logic.

A measurable business result of CCMS is the data quality improvements. Enterprise processes implemented for measuring customer data quality have reported sizable improvements in CCMS data quality over the previous solution. Due to the improved data quality implemented in CCMS, IBM business areas such as sales and opportunity management, and services, which are dependent on accurate and timely customer information, can see increased customer satisfaction. The SOA-based CCMS solution is estimated to provide $8.15M savings per year, mainly through the sunsetting of redundant customer databases that are replaced by CCMS.

Using workflow for its back-end functions, CCMS has enabled a more thorough method to integrate customer information from multiple customer data sources within IBM. This reduced the risk of incomplete or inaccurate customer information, which can seriously delay important business transactions. CCMS now represents IBM's single customer record creation point and master source of customer data. In addition, easier cross references of customer information from multiple line of business enabled new sales opportunities. This also enabled an incremental path to gradual sunset of many redundant customer information data sources.

Best practices used and lessons learned

CCMS is another example of an SOA solution in which the back-end processes play an important role for the management of the information available from the service interface, which represents only the tip of the iceberg. Prior to deploying a service interface, CCMS first needed to create an integrated view of all IBM customer information. This involved leveraging the business process flexibility provided by WBI Server Foundation, to create operational workflows to manage customer data from multiple sources.

As with many enterprise critical applications, a "big bang" approach was unaffordable and disruptive. A Web service to access key customer information was required prior to the development of CCMS back-end processes to manage customer information. A tactical solution was deployed that exposed a service interface to an existing RDc hub. The strategic customer information access interfaces (which CCMS eventually deployed) were utilized for this tactical implementation. This incremental approach minimized the migration efforts from the tactical service to CCMS.

The cost and resources required to migrate from legacy customer data systems to CCMS necessitated non-disruptive and gradual migration path. As IBM has experienced with many other common services designed to meet enterprise requirements, a gradual migration to these services will occur over a period of time and can be accelerated with an aggressive movement to quickly sunset the legacy customer data repositories.

Acknowledgements

We thank the following colleagues for their insight and their contributions by providing cases-study experience reports or other technical input: Geoffrey Meissner and Carl Osipov for collaboration on this series of articles; LeeAnn Kania and Greg Terwilliger, EVS architects, as early SOA pioneers for their EVS technical leadership, and architecture documents that provided a valuable source of information for this article; Bob Fremin, Jim Anderson, Wes Williams, Tim Owings, and Scott Joffe for their role in CCMS strategy implementation.

We also thank many IBM colleagues, consultants, architects, development and project managers, who developed many innovative SOA solutions and took their time to document and shared their experiences and lessons-learned with us. There are too many of them to mention here.

Resources

Learn

Get products and technologies

  • Download IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus ®, Rational®, Tivoli®, and WebSphere®.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into SOA and web services on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=SOA and web services
ArticleID=171943
ArticleTitle=SOA in action inside IBM, Part 2: SOA case studies
publish-date=10312006