“Coming together is a beginning. Keeping together is progress. Working together is success.” -- Henry Ford, American industrialist and pioneer of the assembly-line production method, 1863-1947
Team-oriented approaches for BPM
The recent Women’s World Cup final between the United States and Japan provided an opportunity to witness the success that can be achieved through team alignment. Few thought that Japan had a hope against the Americans. But, as we all witnessed, Japan would not be denied – their triumphant win resulted from the alignment, synchronization, and coordination of the entire team. In a similar fashion, success with business process management requires a similar level of teamwork and collaboration. Success with BPM is maximized when Line of Business (LOB) and Information Technology (IT) stakeholders are aligned in process development and optimization.
In the past, traditional BPM focused on IT-oriented solutions for BPM implementation. BPM tooling suffered due to a technical focus with little involvement of business stakeholders. As a result, business process design and development was defined and executed as a traditional application development project. And, unfortunately, the typical success rates of IT projects are less than optimal. To cite the CHAOS Summary 2009 report from The Standish Group (see Resources), 400 organizations were surveyed and nearly one-in-four (24 percent) IT projects were considered failures. More importantly, an additional 44 percent were considered challenged; for example, they finished late, or over budget, or they did not completely address the requirements. As a result, it can be inferred that applying these traditional approaches for BPM will be subject to similar challenges.
Recently, I was engaged in a client BPM project. The client had spent nearly 12 months in a valiant attempt to build a relatively simple BPM solution; unfortunately they used tools that were heavily oriented toward technical developers. The client tried to apply a business-oriented design approach but developed business processes without fully understanding the business requirements or working with the business users. One year later with nothing to show for the project, the project was cancelled.
Confusing technical design with business design is a common challenge. Success with BPM requires business and IT to work together on a design that realizes common goals and objectives -- and, therein lies the challenge: how does an organization achieve a balanced alignment with business and IT working together to define and develop BPM solutions? A business-based approach to BPM requires a convergence of business and IT roles as active participants in all stages of the BPM development process. Although managing the organizational aspects in collaborative BPM design are daunting, IBM has found that the traditional approaches to BPM development are often the key impediments to success. Without a development and operational approach inclusive of both business and technical stakeholders, BPM continues to be met with implementation challenges resulting suboptimal results. Conversely, an approach with closer alignment of business and IT stakeholders in BPM implementation results in more rapid development with higher satisfaction of business requirements and higher solution quality.
Challenges in traditional BPM approaches
In traditional “old school” BPM development, business requirements were captured from the line-of-business in documents, spreadsheets, Visio, and PowerPoint® slides. These artifacts would normally be reviewed with IT personnel resulting in updated design documents that looked polished, however the actual value of the design artifacts was questionable (though not withstanding the "required" approval signatures from key resources). Based on these updated "requirement" documents, IT would generate subsequent deliverables to document a high-level solution design and business users would review, approve and sign. At that point, IT would commence development with periodic updates to the business via review checkpoints. Depending on the complexity of the solution, a prototype might be created and reviewed by the users. The solution would be updated reactively: typically changes would arise due to poorly communicated and documented initial requirements. Based on this process, it is no wonder that the failure rate on traditional development is nearly one in four… your odds are better on a slot machine.
But there are other factors at work as well when we look at BPM design approaches. A recent survey conducted by the Economist Intelligence Unit (see Resources) shows that 80% of organizations implementing BPM initiatives have faced employee resistance and experienced limited success. The reason for the resistance was tied to two key factors:
- Employees had little or no say in determining the new process (31%),
- New process didn’t map to the way employees thought their jobs should be done (28%).
Using a traditional IT waterfall approach for BPM results in less than successful outcomes due to the aforementioned chasm between business and IT. To make this situation more challenging, business users are typically resistant to change; they would rather perform their job the “same old way” versus employing new approaches. To be successful in BPM, we need to focus on these aspects in the approach to the business process development. Often this issues can be isolated to a failure to capture and define business requirements into a coherent design and implementation.
Remember the Henry Ford quote at the beginning of this article? We can map Ford's three points to a BPM design approach:
- Coming together focuses on process discovery to enable business and technical teams to jointly define requirements and document high level solution design.
- Keeping together defines an iterative development approach with business users involved in the solution implementation. “Keeping together” additionally mandates tooling and a disciplined approach to BPM governance ensuring solution artifacts are managed as a deployment unit and versioned within a BPM repository.
- Working together requires an aligned approach to support lifecycle operational/run time aspects, such as BPM monitoring, management, and process optimization.
In addition to a more focused approach to BPM requirement gathering, BPM development requires an agile and collaborative approach for BPM design and optimization. But just adopting an agile approach is not enough. It is critical to define the key roles for the BPM project (including process owners, business analysts, business process architects, process designers, and integration developers) and to identify skilled resources for those roles. BPM projects are strongly advised to define a project-oriented work breakdown structure that identifies phases such as process discovery, process design and development, implementation and post-implementation activities, and articulates the key responsibilities for specific business and IT stakeholders. It is also recommended that organizations begin to define the end-goals and objectives with a strategy to move from pilot initiatives to project-based BPM and, finally, to an enterprise BPM initiatives/programs.
The goal of collaborative business process management requires an evolved approach to BPM solution implementation. In terms of discussing the tenants of a collaborative BPM approach, we will focus on IBM® Business Process Manager to discuss and review the key technical capabilities. Although other tools are emerging to support some of the capabilities, an overview of IBM Business Process Manager technology and client use cases will enable us to explore the capabilities needed for a comprehensive BPM approach.
Coming together: Supporting process discovery and design
Before we proceed, we need to define what we mean by a process in IBM Business Process Manager terminology. A process provides the container for the process definition, including services, activities and gateways, timer, message, and exception events, rules and variables. Processes are implemented as reusable business process definitions (BPDs) in the Process Designer. With that in mind, let’s take a look at the approach that supports the concept of collaborative BPM development.
Process discovery represents a critical activity in BPM development but is often constrained when business users capture their requirements in Word or Visio and then push this document to IT for analysis and design. This is not a good practice for designing a business-oriented solution. Successful BPM design requires a method as well as tooling to enable business users to capture requirements and actively collaborate with other stakeholders (including IT) to refine these requirements. As an example, IBM’s Blueworks Live™ solution provides a business-oriented web-based solution for process modeling, requirements capture, and documenting key process. Blueworks Live is supported as a managed software-as-a -service solution that is easy to manage for both small and large process design teams.
Recently, I was involved with a client designing an account opening solution. The business stakeholders worked with a system integrator to define the high level solution design. Using IBM's Blueworks Live, the business users were able to quickly develop the milestones and activities for the end-to-end process and to develop draft process models. This capability enabled the business users to design and prototype the envisioned account opening process with minimal assistance from the corporate information technology team. In Figure 1, you see the discovery map in Blueworks Live for this client’s on-boarding (account opening) process. The brown boxes identify the high-level milestones in the process and the blue boxes represent the tasks and activities that correspond to the specific milestones.
Figure 1. Blueworks Live discovery map
This discovery map is then used to create an initial process diagram with definition of swimlanes, decision steps, and the high-level process logic, as shown in Figure 2.
Figure 2. Blueworks Live initial process diagram
The process design can be analyzed to provide insight into process flow characteristics, cycle time, resource requirements, and potential system impacts. More importantly, this design can be directly shared with business and technical users to refine the design and ensure a consistent set of BPM requirements.
Using a collaborative approach to process discovery, this type of planning provides initial artifacts for the development of the account opening implementation. IT can directly link to these process assets in Blueworks Live with the IBM process development tools. Additionally, the process artifacts can be exported in multiple formats including Business Process Modeling Notation (BPMN 2.0) and XML Process Definition Language (XPDL 2.1) to support other process engineering solutions.
Keeping together: Managing process development
After process discovery, the business and technical stakeholders require a collaborative and iterative approach to development to ensure the successful implementation of the process. The evolution of BPMN has been a key enabler in business-oriented solution design by providing visual/graphical modeling which is appropriate for business user review and authoring. But the use of BPMN is only one facet to supporting collaborative BPM design; the BPM tooling also needs to support the rapid prototyping of business processes and the definition and implementation of technical integration components.
The convergence of the acquired Lombardi solution with IBM’s industry leading IBM WebSphere® Process Server provides an example of representative development foundation which directly supports this approach. Figure 3 shows the high-level IBM Business Process Manager solution architecture.
Figure 3. IBM Business Process Manager
The Process Designer (in the upper right) supports the process modeling, prototyping, and testing as part of the BPM development process. The development environment is Eclipse-based using graphical tools and properties sheets to support component implementation. Through drag-and-drop design and filling in associated property sheets, business and technical users can quickly author process artifacts. These artifacts include:
- Business process applications consisting of the processes and services for a solution.
- Business process definition defining the BPMN process flow.
- Toolkits to provide a reusable, sharable set of components used across solutions.
- Snapshots which represent a deployable version of a business process application or toolkit.
- Tracks that enable teams to work in parallel on multiple versions of the same process .
The Process Designer provides support for business rule authoring with a rule authoring experience based on IBM’s ILOG® JRules. Additionally, the rules content can be exported to ILOG JRules to support enterprise business rules management.
As part of the development approach, the Process Designer enables direct playback of BPM solutions to enable validation of requirements and interactive prototype sessions with business users.
Development of technical integration artifacts is accomplished via support for Advanced Integration Services (AIS). AIS components provide implementation assets that are callable from other Process Design services. An AIS interface corresponds to a single operation of a WSDL definition and is exposed by an SCA export of an implementation module. These components can leverage full power of SOA via BPEL-based micro/macro flow constructs, ESB-based mediation flows, integration with IBM’s JCA-based connectors, interface and data mapping, relationships, and other advanced process choreography capabilities.
Figure 4 shows a depiction of the interaction between these components; the “blue cloud” represents a business process invoking an AIS through binding to an interface (as designated by the “I”) to the service. This module is additionally invoking a mediation module and then returns control to the calling process (the blue cloud in the lower right).
Figure 4. AIS components
This approach to supporting business oriented process design and technical integration components as a sharable service promotes both loose coupling and a separation of business and technical considerations. Development of AIS components is enabled via the Integration Designer. IT developers use the Integration Designer to implement the technical integration components (AIS) that are invoked within specific business processes. These assets are managed as reusable components within the IBM Process Center. Integration Designer provides for the implementation of components, including data transformation artifacts, Business Process Execution Language (BPEL) orchestrations, and interfaces to applications and back-end systems. Integration Designer components are assembled in modules and use imports and exports to provide integration between modules. Additionally, libraries provide collections of components that can be shared among integration modules.
Integration Designer provides an ability to support component bindings across many transport/protocol standards, including SCA (Service Component Architecture), Web services via SOAP (HTTP or JMS), JMS, IBM WebSphere MQ, HTTP/REST, EJB/Java™ EE, EIS (Enterprise Information Systems including SAP, PeopleSoft, Seibel, and other key enterprise applications). Integration Designer provides support for testing modules and components on the included local Unit Test Environment. Integration services are synchronized with the Process Center. Additionally, the IBM BPM development tools can directly integrate with Service Registries (such as IBM WebSphere Service Registry and Repository) to provide SOA governance.
The key aspect of collaborative BPM development is focused on providing a robust BPM repository. In the IBM BPM offering, the Process Center provides this functionality managing concurrent access to process applications through locking. The Process Center provides asset management as well as version control of assets across process applications. Through the Process Center console, developers manage the creation and administration of process applications, toolkits, and associated process artifacts. Figure 5 shows the key focus areas for the Process Center.
To be successful in collaborative BPM development, a shared development platform is required with the ability to control or govern the deployment of applications to run time environments. As an example, the Process Center supports the governance and management of process assets and associated dependencies. Process applications include references to one or more toolkits to support sharable/reusable collections of components, such as rules, services, and interfaces. Changes are saved to the Process Center repository as developers implement solutions. Additionally, process application snapshots are deployment units that can be deployed to a remote Process Center or remote run time servers for testing, staging, or production.
Another critical function in BPM engineering is the ability to support the concurrent editing of processes. The IBM BPM solution enables this functionality via parallel workspaces. The Process Center manages the versions of business process definitions and toolkits. This unique versioning approach provides the capability to support “back in time” version management to enable developers and administrators to roll back to earlier versions as required.
Through IBM’s experiences implementing BPM solutions with clients, we have seen that there is a full spectrum of BPM requirements. Many of our clients have advanced requirements, whereas others are just getting started with BPM. IBM BPM supports multiple configurations to match the typical entry points or stages in a company's BPM maturation. The configurations are listed in the table below.
Table 1. IBM BPM configurations
Complete set of business process management capabilities
Configured for typical business process management projects
Configured for first business process management project
Enterprise-level integration requirements normally require the Advanced configuration, which includes the Integration Designer to support enterprise level process and service integration. (Further discussions on the differences between IBM BPM configurations are beyond the scope of this article. See Resources for more information on configurations.)
In closing on the area of BPM solution development, the ability to support both business and technical stakeholders is essential for enabling a team-oriented approach to BPM. In the IBM BPM solution, Process Designer and Integration Designer provide an innovative collaborative platform for BPM development. The Process Designer provides for development of BPM process solutions by business and IT stakeholders with the Integration Designer tooling enabling IT to develop integration components as reusable assets. These capabilities are depicted in Figure 5.
Figure 5. Process Designer and Integration Designer
As a result, business and IT users use the Process Designer to edit process definitions and related assets. This integrated perspective provides a business-oriented view of process solutions without requiring business users to understand or design the technical implementation aspects. Similarly, technical developers use the Integration Designer to provide visibility to relevant business projects and processes to support component development.
Working together: Providing process management and optimization
Ongoing success with BPM initiatives is associated with the ability to optimize processes over time as changes occur via internal and external factors, including economic conditions and regulatory aspects. As a result, it follows that process optimization needs to be detailed during development and design as well as following process implementation as an ongoing part of the process lifecycle. This requirement implies that the BPM method and associated tooling needs to directly support the management, monitoring, and ongoing enhancement of processes. As an example, the IBM approach with Process Center enables ongoing collaborative design and process optimization for the duration of the life of a given business process. The built-in governance and versioning capabilities in the BPM tools, in conjunction with BPM monitoring capabilities, provide for lifecycle management and optimization of BPM solutions. This approach enables business and IT stakeholders to evolve the process definition over time via a shared process model to support to process improvement.
“Working together” represents the closed loop aspects of process lifecycle management. In short, the best way to improve a process is through measurement and then take the appropriate steps to optimize identified bottlenecks. Through a rich set of monitoring tools, process owners and stakeholders proactively work to optimize business processes over time. Within the IBM BPM solution, the Process Portal provides a view of outstanding tasks and visualization of overall resource and process performance. During process execution, the IBM BPM solution gathers and correlates performance events and business data via metrics accumulated in the Performance Data Warehouse.
In addition, the administrative functions in the Process Portal enable end users to initiate and alter processes based on the privileges associated with their specific roles. The Process Portal provides administrators with the ability to manage run time servers. In the IBM BPM Advanced version, these capabilities can be augmented with the IBM Business Space capabilities to provide role-based customized user interfaces, and can be supported in a standalone mode as well as within an IBM WebSphere Portal environment. The below graphic shows a sample of the types of interfaces enabled via the Process Portal environment, which includes dashboards, scorecards, and BPM diagnostics.
Figure 6. Interfaces enabled through Process Portal
The concept of “working together” is enabled via the visualization of performance bottlenecks within the process model diagram. This capability provides process owners to quickly identify potential “hot spots” and recommend process enhancements. The IBM BPM solution also provides a number of optional capabilities for management and monitoring including:
- IBM Business Process Manager for Microsoft® Office to enable users to view and run IBM BPM tasks directly from their Office desktop via an Outlook® plug-in.
- IBM Business Process Manager for Microsoft SharePoint® enables SharePoint users to view and run IBM BPM from SharePoint portal pages.
- IBM BPM solution can be used as part of a comprehensive business activity monitoring solution architecture using the IBM Business Monitor.
The IBM Business Monitor approach provides the ability to aggregate and present information from the IBM BPM solutions as well as other internal and external sources, including B2B solutions, third party applications, and other middleware components. The extended capability in the IBM Business Monitor provides real-time visibility and the ability to visualize and operationally manage extended solution architectures spanning multiple applications and organizational boundaries. The IBM Business Monitor contains a limited license of IBM’s Cognos® Business Intelligence 10.1 platform to support reporting and predictive analytics into these extended solutions. IBM Business Monitor supports a wide range of IBM technologies including IBM BPM, IBM Decision Server, Integration Designer, IBM Industry Packs, WebSphere Business Events, IBM FileNet® technologies, IBM Case Manager, IBM CICS®, WebSphere Adapters, WebSphere Enterprise Service Bus, and other key IBM technologies.
The ability to provide real-time control for in-flight BPM processes and to rapidly identify key bottlenecks in BPM execution is essential for process management. At a minimum, process owners require built-in administrative functions to quickly respond to changes in process performance. More importantly, BPM solutions must allow stakeholders to understand and predict process performance through scorecards and dashboards. Through these functions, stakeholders can better understand process behavior and collaborate to support process optimization over time.
Henry Ford and BPM: Teaming for success
Henry Ford’s vision for team-based collaboration was revolutionary in terms of standardizing automobile manufacturing via the concept of the assembly line. By engaging a team approach to developing automobiles, Ford applied key BPM principles to manufacturing and set a direction for the future. Little did he know that those same concepts would be applied nearly 100 years later as part of collaborative BPM. Ford’s concept of coming together, keeping together, and working together can be mapped directly to a collaborative approach for BPM discovery, design and development, execution, and refinement.
As mentioned earlier, traditional BPM solutions by themselves often provide a less-than-comprehensive approach to BPM. These approaches miss the mark for the majority of BPM projects due to a technology-centric focus that limits the participation of business users in process design and development. On the other hand, BPM is successful when it facilitates business change – and this normally requires business users to be active participants in BPM lifecycle activities. The collaborative approach to BPM management is founded on business and technical stakeholders participating jointly in solution design, development, and optimization.
By adopting these overall principles as part of BPM initiatives, organizations will find that BPM projects will be more successful and BPM solutions will provide higher ongoing business value for the enterprise.
Thanks to my BPM teammates Bertrand Portier and Stuart Jones for their advice and comments on this article – in addition to being comrades in arms, working with these two architects is always enlightening and refreshing.
- Introducing IBM Business Process Manager V7.5
- IBM Business Process Management Journal
- CIO: Recession Causes Rising IT Project Failure Rates
- Business Process Management (BPM) from IBM
- IBM Blueworks Live
- IBM Business Process Manager product information
"Organisational agility: How business can survive and thrive in turbulent times" - Economist Intelligence Unit, March 2009
- IBM developerWorks WebSphere