Introduction

Much has been written about the current business environment and the undergoing dramatic changes: Fierce competition from traditional and nontraditional players, ever-changing regulatory compliance requirements, pressures to create new revenue sources and demands for more innovation and flexibility. To succeed in this environment, enterprises are transforming by rethinking industry structures, becoming componentized and adopting service orientation to achieve desired flexibility.

SOA finds increasingly broad acceptance and is emerging as the dominant technology to support such business transformation, by providing tighter linkage between business processes and enabling IT. While analysts, including Ron Schmelzer and Jason Bloomberg (ZapThink, April, 2006, see Resources), call IBM "one of the foremost pioneers in the movement toward Service-Oriented Architecture" for work with its clients, for the last several years IBM itself is undergoing SOA-enabled business transformation.

Clients often ask us to share our experience inside IBM with them. In this article we provide a glimpse into the first two of seven SOA case studies -- real internal transformation initiatives. Subsequent articles will address the remaining case studies. Each initiative has already demonstrated tangible business results and provides valuable lessons:

• Case study 1: Customer Order Analysis and Tracking System [COATS]
• Case study 2: Microelectronics "factory in a box" [Microelectronics]
• Case study 3: Export Validation regulatory compliance [Export]
• Case study 4: Central Customer Master System [CCMS]
• Case study 5: IBM Intranet Password -- identity management for internal applications [IIP]
• Case study 6: IBM Intranet Password -- identity management for external business partner applications [IIPX]
• Case study 7: Customer web identity management [Web Identity]

These case studies have been selected to represent a wide variety of business challenges that SOA is leveraged to solve. Some of these are commonly occurring cross-industry scenarios, such as case studies five, six and seven. Others, like case study two, are industry specific, yet have typical challenges that can be successfully addressed by SOA.

Those readers who are still asking why they should consider SOA, or who need to create a business case for its adoption, might find Table 1 and the detailed descriptions of business drivers for each initiative helpful.

In addition to business context, for each case study we describe challenges that the initiative had to overcome, an architectural overview of the resulting solution, with its enabling technologies and tools. We also describe the tangible business results achieved by each, and best practices and "lessons learned" that were harvested and are being applied across IBM and with our clients.

SOA value propositions

During the past several years, the authors and their IBM colleagues worked with hundreds of clients to implement SOA-based solutions to solve various business problems. While everyone is talking about business flexibility and agility in general, each initiative is usually driven by a concrete business value proposition -- or desired business outcome. Enterprises adopt SOA as a way to address different business challenges. Desired business outcomes can be represented by several categories described in the table below.

While a successful SOA implementation can achieve several different business outcomes, often there are one or two key drivers that spark an initiative. Although benefits achieved through services reuse leading to development and integration costs reduction are essential, in the long run they are secondary to SOA's business transformation value.

IBM case studies described in the following sections had different desired outcomes, both internal to the corporation and external, as depicted in Table 2.

Case study 1. Customer Order Analysis and Tracking System: From legacy to on demand

Business context

Customer Order Analysis and Tracking System is a shared order entry application for more than twenty manufacturing plants worldwide, each with its own customization needs and access patterns, e.g."High Volume-Low Price" vs."High Price-Low Volume'" A part of the IBM Supply Chain, COATS plays a critical role in order fulfillment, processing a large number of orders and helping to ship over 10,000 parcels per day. With 365x24 coverage, COATS fields orders from IBM customers, business partners and IBM sales professionals for complex configured hardware: System i, System p, System z, Storage, Printers, and Retail.

Buyers indirectly access COATS by introducing orders for new machines, upgrades, customizations, and other changes to existing orders. The application sorts and prioritizes these orders, comparing them against manufacturing rules and the customer's installed hardware base. COATS "translates" the flow of orders into manufacturing material lists (Bill of Materials) and other instructions, which are then forwarded to appropriate IBM manufacturing plants worldwide. The manufacturing plants then fulfill the orders and ship them to the customers' premises. An example of an order will be for a mainframe with detailed specifications, pre-loaded software, delivery date, etc. An example of the corresponding output will include a configuration to the manufacturing floor of a specific plant, in the corresponding nomenclature, including bill of materials, configuration plan, etc.

Challenges

The original application was a complex 25-year old batch system. It included 1.4 million lines-of-code of PL/1, OS/390 Assembler, Java and other programming languages and was running close to capacity at peak times. Batch bottlenecks and conflicting data delayed orders and shipments.

With hard-coded business rules, procedures, logic and data access, COATS has grown in such a way that it could not be easily adapted to address existing and emerging business demands. Changing a business process often required extensive recoding: developers had to understand a complex set of point-to-point connections and component interdependencies to anticipate the effects of changes. To meet frequent order alterations, including automatic alterations performed by the scheduler system to meet customer's delivery date, multiple databases had to be updated and queried, depending on geography and other parameters, such as sold machines history kept for five to ten years.

To support new initiatives, new product and business opportunities, the application was frequently updated. With rigid legacy code, considerable amount of resources (both time and money) was spent on new functionality development - each version took six months to develop and more than 8,000 developer hours.

There was real need to improve access to functionality and valuable business data residing in existing legacy system and to be able to access it easily from other systems.

As with many enterprise critical applications, "big bang" replacement was unaffordable and disruptive.

SOA-based solution

The project to transform COATS focused on several objectives ranked as critical by the business stakeholders:

• Reduce the frequency of manufacturing stops caused by application changes or errors
• Decrease the potential for missed revenue caused by discrepancies in batch processing of orders across COATS subsystems
• Enable a faster and easier way to meet business requirements by speeding up the upgrade cycle and enabling incremental rewrite, change and redeploy functions
• Enable business process management through modeling and monitoring of business process models.

The Order Management Component Services project is transforming the overall COATS system to a real-time order submission system.

The Service-Oriented Architecture of the solution, depicted in Figure 1, standardizes the connections between business processes and IT requirements. In this architecture, the business rules are externalized and legacy system functionality is componentized to promote flexibility, scalability, and reuse.

The Order Process Subsystem is responsible for receiving requests to process groups of customer orders and then forwarding the orders to other services. It can handle multiple channels including business-to-business messaging systems for both real-time and batch processing.

Upon receiving the orders, the Order Process Subsystem marks them for validation and forwards the orders to the Validation and Topology Generation service. The service analyses and uses specialized validation services for different categories of orders.

The groups of manufacturing orders that pass the validation step are then sent to the Manufacture Plant service to be converted to bills of materials and forwarded to the manufacturing plants. This service interacts with service adapters for existing legacy or business partner applications that actually deliver the bills of materials to the destination plant.

The development process started with decoupling of business process and data from enabling IT. Business processes were modeled and represented within IBM WebSphere® Business Modeler (hereafter referred to as "Business Modeler").

The COATS development team adopted an iterative approach to the transformation project. The iterations started with business analysts using Business Modeler to build a model of the "as-is" business process used in the original COATS application. With the model in place, the analysts were able to identify opportunities for improvement, propose changes and estimate a projected impact of the changes on the COATS application. In addition, the analysis of the "as-is" model helped identify existing assets for reuse.

To create the "to-be" version of the model and finalize the specifications for the business processes, services and components the team used the project objectives considered critical by the business stakeholders. These objectives were used to define the decision criteria for identifying areas of the "as-is" model that would most benefit from the transformation to a flexible implementation leveraging workflow and business rules.

To identify and specify the COATS business services, the team adopted a repeatable approach that later contributed to the formalization of the IBM Service-Oriented Modeling and Architecture (SOMA) method. SOMA is a method intended to enable target business processes through the identification, specification and realization of business-aligned services that form the SOA foundation. SOMA creates continuity between the business intent and IT implementation by extending business characteristics into the IT analysis and architectural decisions. The analysis and modeling performed during SOMA is not specific to technology or products, but establishes a context for making technology and product specific decisions in later phases of the lifecycle. Its goal is to provide guidance in the modeling (analysis and design) of SOA. SOMA is used by IBM process engineers and architects on customer engagements and with internal projects.

In parallel with the business analysts, the data architects used Rational ® eXtended Development Environment to define Unified Modeling Language (UML) -based data models. These data models represent business objects to be used by the COATS business processes and for communication across components.

The development artifacts -- Business Process Execution Language (BPEL), Web Service Definition Language (WSDL), XML Schema Definition (XSD) and Java based implementation of the data model produced by Business Modeler and Rational eXtended Development Environment -- were further enhanced by integration developers using the WebSphere Studio Application Developer Integration Edition tool. The development team used WebSphere Studio Application Developer Integration Edition to implement the following changes to complete the workflow implementation:

• Enable access to legacy system by creating adaptors to MQSeries® and CICS®
• Add staff activities for human decision making and exception handling
• Refactor decision points into business rules stored in an external BRBeans repository
• Integrate with a WebSphere Portal presentation layer.

The lifecycle of COATS extends beyond development to cover aspects of business process management. After a development iteration produces a deployable, working build of the application, business analysts can monitor the live performance of the system by tracking key performance indicators (KPIs). The KPIs are defined by stakeholders prior to the implementation of the workflow and give a business view into the system by delivering information such as an average number of orders flowing though the system per hour, number of invalid orders handled by the system and the average time for an operator to resolve an invalid order. Ultimately, the executives can use the information gathered from monitoring of the application to make decisions about changes in business processes to meet business unit performance objectives.

Business results

Several incremental releases of the new SOA-based solution resulted in improved business performance and flexibility. In addition, reduced development cost and faster development turn-around time were achieved. The business results are summarized below:

• Order transaction processing time reduced from 4 minutes to 10 seconds. In COATS, a daily average is approximately 2,500 requests (about 300 at peak hours). Thus, four minutes savings, total to potentially more than 150 hours daily speed-up in handling of orders per day when transformation is completed.
• This solution streamlines integration between systems, enabling real-time transactions that reduce discrepancies in delivery scheduling.
• Ability to make on demand changes to the run time workflow, through easily selectable rules.
• New system eliminates redundancies in the development processes
• These improvements in the development cycle mean that IBM is able to react with more agility to changing business requirements related to order fulfillment processes
• More than 25 percent reduction in development time and cost
• A better error-handing facility included for faster exception processing resulted in revenue savings
• During this work we harvested reusable development assets and best practices. This initiative helped us to harden the SOMA method (see Resources).

Best practices used and lessons learned

This initiative helped us to create, implement and harvest methods for incremental transformation of legacy business functions to real-time, agile, SOA-enabled solution. Best practices used and lessons learned are summarized below:

• Use incremental implementation of services to achieve early buy-in and a non-disruptive migration path while managing expectations. Co-existence of services with legacy code supports the gradual transition of system functions to the new architecture.
• Align business/IT architectures through service modeling.
• To develop agile business processes, address full services lifecycle -- from modeling to monitoring -- supported by methods and tools.
• Follow an iterative design and incremental development employing modeling, design and integration patterns (see Resources).
• Create early BPEL process models
• Do not include activities detail -- they should be documented in use cases
• Create a data model at the same time you create the business process model

We demonstrated incremental transformation through proper use of service modeling, incremental development methods, tools, and middleware components. The process customization, incremental transformation, and legacy integration methods developed in this initiative are being replicated across IBM and with our clients.

You can learn more about how to build reusable assets to transform an order processing system in the developerWorks series "On demand business process life cycle" (see Resources).

Case study 2. Microelectronics "factory in a box"

Business context

The deconstruction (unbundling) of the corporation and enabling it to focus on its core business competencies naturally led to the emergence of collaborating ecosystems. The manufacturing industry discovered the power of specialization and collaboration first and took it to new heights, closely followed by the electronics industry (see Resources).

In this collaborative ecosystem, the semiconductors industry is moving from a vertical integration of completely self-contained manufacturing systems to an integrated participant network of partners, each providing specialized manufacturing services with high efficiency and lower cost. Given today's economic climate to maximize the use of worldwide resources, the operating context is rapidly growing from a focus on local capabilities to a global collaboration of capabilities from multiple countries. The increased complexity of product solutions in microelectronics to meet new customer needs is driving the growth of custom solutions over more traditional standard product offerings. This requires a more adaptable manufacturing system with a greater variety of execution capabilities than what can often be met by completely 'in-house' solutions.

In 2003, IBM Microelectronics recognized the need for more flexibility to react to changing manufacturing requirements associated with wafer and module manufacturing across multiple sites and vendors. The requirement for additional flexibility and responsiveness to create "on demand' manufacturing capabilities was the genesis for the concept of a virtual manufacturing enterprise focused on an integrated approach to manufacturing control and execution across multiple microelectronics sites -- both in-house and outsourced.

Challenges

In mid-2000, IBM Microelectronics experienced challenges similar to the rest of the industry players:

• Financial Models: Although increased competitive pressures are causing individual product unit costs and prices to fall, the complexity and cost associated with new product solutions continue to grow. More complex solutions also increase the complexity of planning, development, and manufacturing processes. Higher levels of demand volatility in today's semi-conductor industry drive increased expenses investment volatility. It is often difficult to predict and prepare manufacturing systems for rapid fluctuations in product demand. Investments to manufacturing infrastructures must be strategically timed to coincide with expected or unexpected market conditions.
• Domain Integration: Although now focusing on core competencies, a sufficient level of semi-conductor product diversity exists that requires the need to integrate historically separate manufacturing systems. This integration requirement also is applied to various business domains, such as Engineering, Enterprise Resource Planning, Test, Opportunity Management, Finance, Release, Product Design and Development, and Information Warehouses.
• Outsourcing: The new globalization trend requires the need to integrate multiple and interchangeable partners. In-house manufacturing functions must be quickly duplicated to outsourced vendors with minimal operations downtime, by building capabilities once and leveraging them. These vendor functions must be integrated for functions such as wafer fabrication, wafer test, module build, and module test. Pathways between all participants must be reliable and secure, and IBM must have visibility to all events as they occur, in addition to being able to collect all data as it is produced. Vendor participants must accept some level of control from IBM for consistent product process results, and must have access to all necessary functions -- even if not local/native to their environment. A global, virtual manufacturing architecture requires the use of open, industry standards to increase interoperability and minimize the time for partners to adapt to IBM's interfaces.

SOA-based solution

To address the above-described challenges, IBM's System Technology Group (STG) semiconductor manufacturing created solution based on an integrated modularized architecture. Their goal to create a virtualized framework for the management of internal manufacturing resources coupled with an efficient outsourcing program led to a novel concept that they called "factory in a box". This is a logical, modular "box' of manufacturing features that include network transport, security, data management, monitoring, manufacturing controls, and unique customizations. The "factory in a box" is realized through the packaging of these features into a set of IBM software products and customized code, which is also referred to as the Multi-source Data Integrator (MDI). The MDI is installed both on IBM and vendor partner premises, as depicted in Figure 2, to create a seamless interaction of STG's various manufacturing business domains to physical manufacturing installations, regardless of their location -- thus creating a virtualized manufacturing environment. This allows vendors to be viewed as an extension of the IBM manufacturing environment.

One might ask "ok, nice story about On Demand business, but where's the SOA?" Key MDI integration elements are its Enterprise Service Bus, through the Business Modeler Message Broker, and a set of self contained SOAP based Web services. The Message Broker, following the established ESB architectural pattern, provides for a high degree of decoupling through protocol and message translation, in addition to task routing. Some of the services are completely self-contained to support only internal MDI function, and are transparent to anything outside of the box. Other services, however, are accessible by IBM application consumers, vendor application consumers, or sometimes both. It's this packaging of SOA features in a portable form has prompted several to rename the original label of the MDI to "Services in a Box".

A sampling of services:

• Wafer Die Disposition: Various functions related to wafer manufacturing and test. They were originally targeted for outsourced manufacturing, but are increasingly being used internally.
• Manufacturing Execution System Transaction: Provides neutral interface to numerous manufacturing execution systems, enabling manufacturing lot status and attributes to be queried.
• Module Disposition: Supports various functions involving module manufacturing and test
• ULL Generator: Functions to support rules-based generation of unique, site-specific universal lot labels (ULLs). ULL Service is typically customized for each unique MDI location to generate ULLs using site-specific rules.

The MDI is built on a foundation of proven IBM middleware and operations management products. These products include DB2®, WebSphere Application Server, MQSeries®, WebSphere Business Integration Message Broker, WebSphere Business Integration Connect, and Tivoli Management Environment® for Monitoring. It is installed in both IBM and vendor premises by IBM Global Technology Services. The vendor installation is located in a demilitarized zone on an IBM managed server, and is accessible to both IBM remote communications and through the vendor's firewall for vendor access.

Architectural solution highlights:

• Rapid Outsourcing: A common, standards driven mechanism to interact with both IBM internal and vendor manufacturing facilities, which facilitates an efficient outsourcing of IBM's manufacturing
• "Services in a box": A novel concept that allows a predefined set of configurable services to be easily transported and managed at remote locations
• Externalized business rules: Rules engine in the MDI is separated from operational code, for increased flexibility to adjust to new business conditions without recoding
• "Virtual Factories": These are enabled with a technical Isolation layer framework between Business Domains (each with their respective sets of processes and data), and the black box implementation of the MDI
• Services: Decomposition of new and legacy function into reusable services

Business results

This solution created a "virtual" manufacturing system for IBM Microelectronics involving multiple, interchangeable, and integrated partners (vendors) -- a "factory in a box" enabled via SOA.

As an example, the Technology Group's outsourcing of a semiconductor testing facility to Amkor Technologies resulted in the IBM system being down for just four hours during the migration from IBM, which may normally have required three days before the implementation of their new SOA enabled architecture. Rules were updated to route transactions to the new facility, and the changeover was largely transparent to normal operations. This is the result of their modularization techniques implemented by the MDI's "factory-in-a-box" construct. In addition, this "pre-packaged" SOA architecture allows for outsourcing planning to now be accomplished in a more turnkey fashion rather than customized for every new opportunity. This enables a planning schedule of just a couple of weeks rather than the three months previously required.

Best practices used and lessons learned

Several important lessons were learned during this initiative:

• Virtualization and isolation of business domains with related processes and data created an "isolation framework" for an optimal decoupling between domains, which enabled "Virtual Factories"
• Externalized business logic contained in a rules engine separate from the primary operational code and services allowed for faster business process updates
• Access to tool beta versions can help define tooling requirements for future projects
• Completing SOA pilots prior to initiating projects can help infuse the technology more efficiently and reduce the impact on project schedules.

Acknowledgments

We thank the following colleagues for their insight and their contributions by providing cases-study experience reports: Jim Doran, German Goldszmidt, and Dick Panko.

A core team in IBM Technology Group defined the strategy, architecture, and implementation of Case study Two: Microelectronics "factory in a box". The primary people responsible for this excellent example of SOA within IBM are Steven Pike, architecture manager, Mike Maslack, strategist, and Dave Lutton, development team lead.

We also thank many IBM colleagues, consultants, architects, development and project managers, who developed described the innovative solutions in this article, and took the time to document and share their experiences and lessons learned -- both best practices and anti-patterns. There are too many of them to mention here.

Resources

Learn

• In "Impact of service orientation at the business level", (IBM Systems Journal, Volume 44, Number 4, 2005) L. Cherbakov, G. Galambos, R. Harishankar, S. Kalyana and G. Rackham describe the important role played by componentization and by service orientation.
  
  
• ZapThink provides advice on the architectural and organizational changes brought about by the movement to XML, Web Services, and Service Orientation.
  
  
• IBM to Help Customers Focus on Growth Through Business Transformation, from the IBM Press room, describes IBM's service to help companies build capabilities that support business goals, while freeing up currently overstretched IT budgets to focus on growth opportunities.
  
  
• Patterns: Service-Oriented Architecture and Web Service, an IBM Redbook, describes how the Self-Service and Extended Enterprise business patterns, and the Application Integration pattern, can be used to start implementing solutions using the service-oriented architecture approach.
  
  
• "On demand business process resource life cycle" (developerWorks, October, 2004) presents a methodology to develop agile, on demand business processes.
  
  
• "SOA Anti-patterns" (developerWorks, November, 2005) explores different SOA antipatterns, which are descriptions of commonly occurring situations or solutions that generate decidedly negative consequences.
  
  
• SOA and Web services -- hosts hundreds of informative articles and introductory, intermediate, and advanced tutorials on how to develop Web services applications.
  
  
• Get the RSS feed for the "SOA in action inside IBM" series.
  
  

Get products and technologies

• Get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®. You can download evaluation versions of the products at no charge, or select the Linux® or Windows® version of developerWorks' no-charge Software Evaluation Kit.
  
  

Discuss

• developerWorks blogs -- get involved in the developerWorks community.
  
  

About the authors

Comments (Undergoing maintenance)

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Introduction
• SOA value propositions
• Case study 1. Customer Order Analysis and Tracking System: From legacy to on demand
• Case study 2. Microelectronics "factory in a box"
• Acknowledgments
• Resources
• About the authors
• Comments

Next steps from IBM

Rational Tester for SOA Quality is an automated web services testing and SOA testing tool that supports Web services SOAP, HTTP(S), JMS, WS-Security, UDDI, and RESTful standards.

---

• Try: The no-charge Rational Service Tester for SOA Quality helps quality assurance professionals validate web service based SOA application..
• Article: See the whats new with Rational Service Tester for SOA Quality, and its Generic Service Client, which provides a single client to interact with any SOA service type.
• Tutorial: Get hands-on experience with this Transport for SOAP feature tutorial using Rational Tester for SOA Quality.
• Buy: Rational Service Tester for SOA Quality

My developerWorks community

Dig deeper into SOA and Web services on developerWorks

Rate a product. Write a review.

Write a review for Rational products today!

Special offers