Iterative development in WebSphere Business Modeler and WebSphere Integration Developer (Marc Fasbinder, developerWorks 2008) describes how to keep business models in WebSphere Business Modeler (hereafter called Modeler) and WebSphere Integration Developer (Integration Developer) consistent with each other as they evolve over time. Based on observation of real-world environments, this article describes iterative approaches that apply to business processes and the capabilities available in Modeler and Integration Developer for facilitating iterations. After describing the possible approaches, we recommend the approach that we believe is best, taking into account the current capabilities of the tools. We'll also walk through three iterative development scenarios to explore the iterative development capabilities of both products and suggest the usage patterns that are likely to produce the best results.
Iterative development is a software development approach that enables the convergence of a solution with the desired goals through successive refinements of requirements and development artifacts. The term iteration is often erroneously used as a synonym for increment. Although iterative approaches may be used with incremental delivery methods, the two are not synonymous. In this article, we talk about iterations as refinement cycles, and not necessarily as deliverable increments.
In the sense that it is a refinement cycle, an iteration does not have to include all phases of the development process. For example, it's possible to have phase-specific iterations, such as analysis iterations or design iterations. As you can imagine, this can lead to considerable complexity and variability in the iterative development method. At the top end of the complexity spectrum, you could end up with refinement cycles within refinement cycles, each with different iteration frequencies. For this reason, it's important to identify the iterative development approach that is best suited for a particular type of software solution, taking into account the capabilities of the tools that are being used to develop it.
BPM and iterative development
Business process management (BPM) solutions are an especially volatile type of business software. A rapidly changing business environment, as well as continuous process improvement practices, imply frequent business process changes, as well as a shorter time-to-market expectations. Since the initial conception of a BPM solution is not likely to be any closer to the desired final result than in most software solutions, this means that more iterations will have to be crammed into shorter periods of time. This is not possible without tools that explicitly support iterative development.
Modeler is IBM's tool for modeling business processes and Integration Developer is the tool for implementing those processes and the integration logic that supports them. Together these tools are capable of supporting an iterative development process for the creation and continuous improvement of deployable business processes. Modeler targets mainly the analysis and, to some extent, the design components of the development process, while Integration Developer is responsible primarily for the implementation aspects. This article discusses iterative development using these two products in a BPM context. We'll designate the activities that involve the use of Modeler as modeling activities (which could include analysis and design) and those that involve Integration Developer as implementation activities. Thus, when we refer to modeling iterations, we're referring to refinement cycles that occur in Modeler, and when we refer to implementation iterations we're talking about the same concept involving Integration Developer.
Iterative development choices
In BPM solution development, the use of tools that support iterative development is a necessary but not sufficient condition for success. The actual iterative development process is equally important. There are many variations of the iterative method. A useful classification of such variations is one that focuses on the stage or stages of the development process where the majority of the refinement effort takes place. Using this type of classification and based on our experience with development processes involving both Modeler and Integration Developer, we've identified four main iterative development approaches:
- Implementation-centric with periodic model updates
- Parallel iterations on model and implementation
The implementation-centric approach consists of the repeated enhancement of implementation artifacts based on technical requirements and, in many cases, on changing business requirements. There is an initial modeling phase and implementation artifacts may initially be generated from the model. However, while the modelling activity may continue, all new refinements are done in Integration Developer, without the benefit of the code generation and synchronization capabilities available in the tools, as shown in Figure 1.
Figure 1. Implementation-centric approach: most iterations take place on the implementation side
We've come across a number of justifications for this approach, including:
- "We have a tight deadline and we can't sit around and wait for the models to be completed."
- "Synchronization and merging are complex to do and we're afraid of making mistakes that will result in loss of work."
- "The generated code is not to our liking, so we have to do a large number of implementation changes, which makes keeping the model synchronized with the application very difficult."
- "We like to keep the business model at a very high 'business' level and implement all the details in Integration Developer."
- Little or no model/implementation synchronization effort.
- It's easier to adopt specific implementation designs that favor certain non-functional aspects, such as performance.
- The business model should evolve to reflect the changes to the implementation, but in reality this doesn't always happen, and if it does, it's unlikely that the implementation will be an accurate reflection of the model.
- The ability to merge model changes into the implementation, as well as the benefits that may be derived from code generation, is lost.
- Analysis is often bypassed and implementations are more likely to not meet the business objectives (resulting in more iterations).
Implementation-centric with periodic model updates
This approach is also implementation-biased, but differs from the previous approach in that the development process explicitly requires that changes be propagated back to the model at regular intervals, or after a certain volume of implementation refinements. Implementation changes are communicated, with partially automated assistance from the tools, to the business analyst. Such an approach works best when the volume of changes in each batch is small.
Figure 2. Implementation-centric with periodic updates of the model
The justifications for this approach are similar to the previous approach, but there is a greater concern with keeping the implementation in synch with the model.
- As in the previous approach, it's easier to adopt specific implementation designs.
- There is a greater level of synchronization between the models and the implementation.
- Even with assistance from the tools, applying changes to the model can be very time-consuming, especially when the batch of changes is large.
- Often the model changes that result from the implementation changes are not well understood by the business analyst.
- Subsequent synchronization with the implementation may reach a level of complexity at which it's no longer feasible to continue with code generation and automatic synchronization of the generated artifacts with the implementation.
- As in the first approach, analysis is often bypassed.
Parallel iterations in the model and the implementation
In this case, the refinement cycles take place in both the model and the implementation at the same time. In both cases, these cycles can result from new requirements, correction of initial mistakes, various types of structural improvements, and so on. At some point, the the model and the implementation are synchronized. Again, synchronization works better when the number of changes is small.
Figure 3. Parallel iterations in both the model and the implementation
"Round-trip" proponents often favor this approach because, among other things, it covers those situations where the implementation needs to be changed swiftly to address production problems or to meet tight deadlines. It's also useful when dealing with existing artifacts.
- This approach allows for the model and the implementation to be refined at different rates, thus reducing their dependencies on each other.
- Existing implementations (that were not initially modeled) can be added to the model.
- It's possible to achieve a greater level of implementation/model synchronization than with the implementation-centric approach.
- Synchronization of the model and the implementation can become very complex, especially when there are a large number of conflicting changes.
- Effort is often duplicated.
- There is the danger that changes introduced during implementation will not be subject to formal analysis and governance processes.
In this approach, all refinement cycles resulting from new requirements or changes in design are carried out in the model. After an (ideally) small number of changes, the corresponding implementation artifacts are generated and merged into the existing implementation. Please note that the model-centric approach does not prescribe that all implementation artifacts be generated from the model, nor does it negate the existence of artifacts or iterations that are specific to the implementation domain. Instead it mandates that all artifacts that are business-relevant and only those that are business-relevant be modeled, so that there is little or no need to propagate implementation changes back to the model, as shown in Figure 4.
For example, in a particular business process, it's necessary to specify that the process has a "notify client via e-mail" activity. The fact that this activity is implemented using the xxx e-mail adapter that supports the yyy interface doesn't add any value to the understanding or improvement of the business process, so this information should not be included in the business process model. On the other hand, if the "notify client via e-mail" activity is not specified in the business process model, there is no way for the developer implementing the process to know about it, so the resulting implementation won't meet business requirements. In this sense, "notify client via e-mail" is a business-relevant concept, while the specific type of email adapter used in the implementation is not. This typically implies that the majority of iterations, and the most significant ones, take place in the modeling domain.
Figure 4. Model-centric: iterations take place in the model
Model-driven development advocates are usually comfortable with this approach, because it places more emphasis on the model, which is understood to be more enduring and "closer to the business" than the technology-dependent implementation.
- Greater control and governance over change cycles.
- Errors are usually less expensive to correct than in implementation.
- Ensures that necessary analysis work is not bypassed.
- Typically, best-practice approaches are built into the code generators.
- Implementation work is reduced and implementations are more consistent and less error-prone.
- Implementation developers have to wait for the completion of each modeling iteration, though this can be minimized by careful planning and by adjusting the ratio of modelers to implementers.
- The generated code may not always meet exact implementation requirements.
- Requires a good understanding of the implementation-specific design capabilities available in Modeler.
The recommended approach
Even without considering tool capabilities and limitations, it's easy to justify a preference for the model-centric approach based on the advantages and disadvantages of each approach outlined above. Of course, our case for model-centric iterations becomes stronger when the tooling capabilities favor this approach. BPM tools, such as Modeler, are good at generating implementation artifacts from the business process models. When you look at generating modeling elements from implementation artifacts, it's a different story. Limited capabilities can be made available, but such capabilities are hard to develop and often can't be fully automated. For this reason, its much easier to keep the models and implementation in sync when all changes (that need to be modeled) are modeled before they are implemented.
Model-centric iterative development is therefore our recommended approach for the development of BPM solutions. This approach results in many benefits, including:
- Governance – facilitates the control and authorization of changes prior to implementation.
- Model and implementation synchronization - unidirectional synchronisation requires less effort and ensures minimization of conflicts.
- Productivity – higher degree of code generation saves time and enforces best practices from the start.
- Solution quality - promotes "analysis-first' habits that are more likely to ensure fulfilment of business requirements.
- Better business and IT alignment - structural equivalence between business process models and implementation is more likely with this approach.
Figure 5 shows a detailed example of a model-centric iterative process using Modeler and Integration Developer.
Figure 5. Iterative development process
This process shows that the model is refined as a result of new requirements and improvements. In addition, there is a feedback link from implementation back to the model to cover the real-life cases where:
- New requirements or constraints are discovered during implementation.
- The generated implementation is rejected because the model is incomplete or incorrect.
- A certain model design is favored for implementation.
- Existing services or data definitions are imported into Modeler for reuse.
Iteration scenarios with Modeler and Integration Developer
Before we start elaborating on our iteration scenarios, let's take a quick look at the relevant capabilities of Integration Developer's synchronization wizard, which is a key enabler for the iterative approach using Modeler and Integration Developer. The synchronization wizard enables the following:
- Addition or merging, as a unit of work, of implementation artifacts resulting from business model changes.
- Deletion of implementation artifacts resulting from business model changes from the Integration Developer developer's workspace.
- Choosing between model and workspace changes when there is a conflict.
- Accepting or rejecting Integration Developer workspace changes.
Here's a summary of how the wizard works:
- An Integration Developer project interchange file (PI) is exported from Modeler. This file contains Integration Developer artifacts that were generated from the business process models.
- When this file is imported into Integration Developer for the first time, an image of the imported Integration Developer modules is created. This image is called a baseline.
- Every time the model and the implementation are synchronized (via the wizard) a new baseline is created.
- When changes are made in Modeler and a new PI file is exported, the wizard compares the contents of this file with the last baseline to determine what modeling changes have occurred since the last baseline was created.
- The wizard also compares the relevant contents of the workspace with the baseline to determine what workspace changes have occurred since the last baseline was created.
- The user accepts or rejects these changes and the accepted changes are committed to the workspace.
- A new baseline is created.
The scope of the synchronization is the selected Integration Developer module and its dependent libraries. The scope of the Modeler changes that participate in the synchronization is determined by the exported PI file. This file may result from the export of a specific Modeler element, for example, a process, or from the export of the entire Modeler project and its dependencies. The wizard works best, that is, requires less human intervention, if the latter approach is employed.
The synchronization wizard can also be used to synchronize the workspace with assets contained in a Rational Asset Manager repository to enable governance of changes. In this case, the wizard behaves in the same way as when you synchronize with a "direct" Modeler export file.
The following scenarios will give you further insight into how the synchronization feature works and illustrate how you can use the recommended model-centric approach with Modeler and Integration Developer. The scenarios aren't intended to provide a comprehensive description of the synchronization capabilities. For more information on this topic, check out Modeling iteratively with WebSphere Integration Developer in the WebSphere Business Modeler V6.2 Information Center.
Note: We used V188.8.131.52 of Modeler and Integration Developer in these scenarios. However, the information still applies to V7 of these products. The most important enhancement introduced in V7 is the capability to visualize the impact of changes graphically.
Scenario 1: Changes in Modeler only
Luke, the business analyst, creates a Search Customer process model, as shown in Figure 6, in the Customer Validation project in Modeler. This model contains only processes that are destined for implementation in the Customer Validation implementation module in Integration Developer. The fact that both projects have the same name and contain the same processes helps Luke in his communication with Lucy, the Integration Developer developer. It also facilitates the automatic synchronization of both projects. Once Luke is happy with the Search Customer process, he exports the model to Integration Developer. Because he wants to simplify the iterative development process, he exports the entire Customer Validation project. He uses the recommended export option, which allows a cleaner separation between specification and implementation.
Figure 6. Search Customer process
Lucy imports the project into Integration Developer and realizes that there are some changes that she wants to make. Lucy could make the changes in Integration Developer, but she knows that these are business-relevant changes that must be specified in Modeler so, in compliance with the company's development process, she communicates her changes to Luke: "I think we need to create a new customer record in those cases where we don't find any customer who matches the search criteria. That's something we have in the old system. Is this also a requirement going forward?"
Luke decides that Lucy's requirement is relevant and he makes the changes. He also adds a change of his own. He changes the name of the map element "Map" to the more meaningful "Copy address status." The process model now looks like Figure 7. Luke exports the model and gives the PI to Lucy, who synchronizes her Integration Developer workspace with the Modeler export.
Figure 7. Modified Search Customer model
Lucy analyses the new changes in the synchronization wizard, as shown in Figure 8.
Figure 8. Modeling and implementation changes in the synchronization wizard
In the left pane, Lucy sees the artifacts affected by the changes. Since she's on the Modeling Artifacts tab is selected, she sees the Modeler changes (hence the familiar Modeler icons). On the right, Lucy observes the modeling changes corresponding to the artifact selected in the left pane. These are called modeling changes because they result from changes in the business process model, but they are, in fact, descriptions of the Integration Developer artifacts that will be added (plus decorator on the top left), changed (delta decorator) or removed (red X decorator) from the Integration Developer workspace. (For more information on icons and decorators used in the synchronization wizard, refer to the WebSphere Business Modeler V6.2 Information Center.)
All the changes made by Luke are accepted by default and this is shown by the green "tick" decorator at the bottom right of each change icon. Lucy has not changed her workspace, so there are no workspace changes and, of course, no conflicting changes.
Before committing the new changes, Lucy takes one last look at the high-level workspace artifacts that will be affected by the changes by selecting Workspace Artifacts in the left pane, as shown in Figure 9.
Figure 9. The workspace artifacts pane showing the artifacts affected by the changes
Lucy sees all the affected modules and, by selecting them one by one, she's able to see what changes will be applied to each module. Lucy understands how the workspace artifacts will be impacted by the changes, so she commits those changes. The changes are successfully incorporated into her workspace.
This scenario illustrates a simple "create a process model in Modeler, generate the model, change the model, and generate again" case. For the most part, this type of synchronization is fairly straightforward and only requires the acceptance of the changes (all Lucy had to do was click Commit). Because Lucy had not made any changes in Integration Developer, she could have achieved the same results by deleting the original Integration Developer project and importing the changed model. However, in that case we wouldn't have had a simple scenario to show you the basic workings of the synchronization wizard.
A more realistic variation of this scenario is one where an implementation has changed; for example, a developer has coded the implementation of a generated Java™ component (stub). In such a case, deleting the workspace would not be a good option.
Note: Pay special attention to the renaming of model tasks that have a Java implementation. In this particular case, a new, empty Java component is generated. However, the old component is not deleted from the workspace, thus allowing you to reference the previous implementation.
Scenario 2: Changes in Integration Developer only
This scenario looks at the synchronization implications of making Integration Developer-only changes; that is, changes that are not business-relevant.
Lucy realizes that there is a missing step in the process required to notify the customer via e-mail. She knows that this activity is business-relevant and she intends to communicate this requirement to Luke. In the meantime, she decides to use the Integration Developer e-mail adapter to implement this service. She creates a Notification Services project and creates the e-mail adapter in this project.
She also makes a change to the way the address status is mapped. Normally she would ask Luke to make the change in Modeler, but there are certain Integration Developer mapping capabilities that the current version of Modeler doesn't support. Lucy performs a synchronization with the latest Modeler export. Because the model has not been changed, there are no modeling changes; only workspace changes, as shown in Figure 10. Lucy realizes that the modules that she created for the e-mail adapter are not shown in the synchronization wizard. This is because the synchronization mechanism only considers changes that belong to the module that was selected for synchronization (CustomerValidation, in this case) and its dependent libraries.
Figure 10. Only workspace changes for the selected module are shown
Lucy sees the change that she made to the map (Assign activity) and, by default, it is accepted. She is careful not to reject the change (by pressing the icon with the blue x decorator) because she knows that doing so will cause her workspace changes to be deleted. "That's one way of reverting to the previous synchronization state," she thinks, but this time that's not what she wants. She commits the change and decides to do another synchronization with the same Modeler export. Figure 11 shows what she sees.
Figure 11. Accepted workspace changes reappear as rejected modeling changes
Lucy sees the same change, but now it appears as a modeling change. Furthermore, the change is now a "delete"(x) from the workspace, whereas before it was an "add" (+). Lucy is perplexed. What does this mean? When she did her previous synchronization a new baseline, which included the new workspace elements, was created. The Modeler export (PI file) doesn't contain these elements, so in order to to achieve a synchronized state, the wizard reports this as a modeling change and suggests that these elements be deleted from the workspace. In V7, these types of changes are classified as implementation changes not incorporated into the model.
Lucy notices that the change is rejected by default. This means that if she commits this change it won't affect her workspace. In other words, her change to the map won't be deleted, which would have been the case if she had accepted the change. Because this change will never be implemented in Modeler, it will continue to show up as a rejected modeling change in subsequent synchronizations.
Lucy now turns her attention to the new module that she created for the e-mail adapter. She synchronizes it with the Modeler export. Lucy is now able to see the remaining workspace changes, as shown in Figure 12. Unlike the previous case, there is no Integration Developer baseline for comparison, so these changes appear as "delete" modeling changes that are accepted by default. She is careful not to commit these changes because they would result in her e-mail adapter changes being deleted from the workspace. Instead she cancels out of the wizard.
Figure 12. Synchronizing a module that was not initially generated from Modeler
Scenario 3. Changes in Modeler and in Integration Developer
One of the fundamental rules of our chosen approach is that business-relevant changes must be done in Modeler. In practice, however, there may not always be a clear separation of implementation artifacts that result from such changes and those artifacts or changes that are implementation-specific, resulting in synchronization conflicts. The following scenario is a more realistic one that provides insight into how to approach synchronization when you have both model changes and implementation changes, some of which may be in conflict.
Lucy tells Luke about the e-mail notification change. Lucy also asks Luke to change the FindCustomer Modeler task to the corresponding Telecom Operations Content Pack (TOCP) findCustomer business service that is a standard for her company. This means that she won't have to update references (or reject changes) with every iteration.
Luke changes the model, as shown in Figure 13, and exports it as an Integration Developer PI file.
Figure 13. The revised model using the findCustomer business service
Lucy uses the synchronization wizard to merge Luke's modeling changes with her workspace changes. The wizard first prompts her about the TOCP modules and she chooses to import them into her workspace, as shown in Figure 14.
Figure 14. Importing new modules into the workspace
Next, Lucy looks at the synchronization page of the wizard, shown in Figure 15, and sees that there are conflicting changes. She had forgotten that she had made a change to the preferred interaction style property of the FindCustomer interface. Now this change is in conflict with the interface being deleted as a result of the replacement of the FindCustomer interface in the model.
Figure 15. Conflicting changes shown in the workspace changes pane
By default the conflicting modeling change is rejected, and this is not what Lucy wants. She accepts the change and the conflicting workspace change is automatically rejected. Lucy is also able to see the conflicting change (in inverse order) in the modeling changes and workspace changes panes. She commits the changes, which are successfully incorporated into her workspace, as shown in Figure 16.
Figure 16. Assembly diagram after accepting the changes
Lucy will still have to wire the the Notify customer by e-mail import to the e-mail adapter and a set of related changes will be shown as rejected modeling changes with each synchronization.
Refining the iterative process
If you studied the synchronization screenshots for the scenarios above, you will have observed that even small changes in Modeler can result in many Integration Developer workspace changes. This is to some extent unavoidable given that the model is specified at a higher level of abstraction than the implementation. With potentially a large number of changes being reported, the synchronization wizard attempts to make the right decisions, and as you saw, only the last scenario, where there was a conflict, required that the user explicitly accept or reject a change.
The above scenarios illustrated the basic behavior of the synchronization wizard. Table 1 summarizes this behavior.
Table 1. Behavior of the synchronization wizard
|Modeling changes||Changes are applied to the workspace.||Changes are not applied to the workspace, but will continue to show up as rejected modeling changes with each iteration.|
|Workspace changes||Changes are applied to the workspace and will be shown in subsequent iterations as rejected modeling changes, unless the model is updated in such a way that it is brought back in synch with the baseline. In these changes, the action is reversed. For example, adding an artifact to the workspace results in a modeling change that, if accepted, will cause the same artifact to be deleted from the workspace.||The changes made to the workspace are deleted.|
Looking at this table, we can draw some important conclusions:
- You must take care when rejecting workspace changes because this may result in loss of work.
- Loss of workspace changes can also result if, in subsequent synchronizations, the corresponding modeling changes are accepted.
- The set of rejected modeling changes can become very large as they will accumulate with each iteration. To keep this set as small as possible, in all cases where the implementation change has a model counterpart, it's important to change the model as well. Making the model changes first results in less synchronization effort.
- The scenario that results in less effort and smoother iterations is when all modeling changes are accepted and when the number of workspace changes is small.
- It also helps if the number of changes that you're dealing with in each iteration is small.
These conclusions, as well as our observations from the realization of many practical scenarios, lead us to suggest the following guidelines to further enhance the recommended iterative process:
- The business analyst and the Integration Developer developer will benefit from working together to ensure that the business process model is implementation-ready and that all the attributes relevant for the implementation mode (for example, WebSphere Process Server) are correctly set. It is, therefore, important that the Integration Developer developer be skilled in the relevant features of the Modeler tool set. This will also increase the likelihood that the modeling changes are accepted.
- The Modeler project that is being synchronized should not contain elements that are incomplete or not destined for implementation. This means that such elements will not have to be rejected when synchronizing.
- Integration Developer-only artifacts that are not referenced by generated artifacts should, as much as possible, be implemented in separate implementation modules. This reduces the set of workspace changes shown in the synchronization wizard.
In this article, we recommended a model-centric approach to iterative development that, when applied correctly, will yield many benefits. We've also shown how you could use such an approach with WebSphere Business Modeler and WebSphere Integration Developer, and have have provided additional guidelines to ensure a productive iterative process given the capabilities and limitations of the tools.
- Iterative development in WebSphere Business Modeler and WebSphere Integration Developer (developerWorks 2008): This article looks at how you can keep the business models in both tooling environments consistent with each other as the models evolve over time.
- Model Synchronization between WebSphere Business Modeler and WebSphere Integration Developer V6.1.2 (developerWorks 2009): This article illustrates the closed cycle development between WebSphere Business Modeler and WebSphere Integration Developer V6.1.2.
- Modeling iteratively with WebSphere Integration Developer: This topic in the WebSphere Business Modeler V6.2 Information Center provides details on how to make changes to a process model and integrate those changes into the implementation, as well as how to selectively update the process model with changes made to the implementation.
- Iterative and incremental development explained (2007, Kevlin Henney)
- developerWorks BPM zone: Get the latest technical resources on IBM BPM solutions, including downloads, demos, articles, tutorials, events, webcasts, and more.