Applying DevOps concepts to your IBM Business Process Manager project
DevOps, a software methodology that integrates application development and IT operations, continues to gain momentum in IT shops. But there is still a gap in many organizations between talking about DevOps and implementing it effectively. This tutorial provides a brief overview of DevOps and looks at how DevOps concepts can be realized on an IBM Business Process Manager (BPM) project.
What is DevOps?
DevOps is a combination of development and operations: an approach to the software delivery cycle that takes a holistic view of the entire approach. At the simplest level, DevOps tries to answer the question "What is the most efficient way to get a good idea into production?"
While DevOps as a technique is relatively new, the idea can be tied back to Edward Deming, who said you should "break down barriers between departments" and allow all departments to work together as a team. Working as a team not only helps eliminate problems that are encountered in production, but helps all departments work more effectively together in the entire service lifecycle, from design through deployment and operations.
So what does this really mean? DevOps starts and finishes with the customer. Customer demand starts the development cycle. Planning is the initial step to determine how customer demand will be met. Business needs to be able to react quickly to customer feedback. Lean and agile techniques facilitate the ability to react quickly. For example, one principle of the lean approach is to deliver as fast as possible. One way to do this is to avoid big, up-front design, which might have to be reworked later, causing delays. Instead, the team can capture requirements for a feature on a just-in-time basis as the team is ready to build the feature.
Development follows planning. Again, agile techniques are important for DevOps, especially short iterations leading to something that's ready to deploy quickly. Techniques like continuous integration, automated testing, and continuous testing are part of a DevOps approach.
Then, finally, it's time to deploy. A DevOps approach uses continuous deployment, the capability to move code into production in an efficient, automated way. Continuous deployment implies a level of automation to facilitate the efficiency.
Dissect an IBM BPM project from a DevOps perspective
Now that you have a basic understanding of what DevOps is, consider how this fits into an IBM BPM project. Look at the overall approach, starting with the discovery phase and making your way through to your production deployment.
The discovery and macro design phases
Prior to the start of an IBM BPM project, an important question must be asked: "What process are we going to implement?"
Using a lightweight approach is the key to answering this question. A process inventory quickly identifies which processes are ripe for implementation. Using criteria that focuses on customer value helps prioritize processes and the selection of a process or processes to implement based on the customer value it can deliver. This kind of value targeting is a lynchpin of the Smarter Process Method, which includes conducting a process inventory, creating a roadmap, and conducting variation analysis. Contact IBM Services if you want to learn how to use the Smarter Process Method with your project.
After a process is selected, there is more work from a DevOps perspective. An IBM BPM process doesn't sit in isolation. It integrates with other systems. Part of the discovery phase should include identifying those systems and the services they provide. The operations team can answer questions about the services including service level agreements (SLAs) and quality of service (QoS).
“"Including a DevOps approach in capturing requirements requires IBM BPM analysts and architects to envision beyond the 'as-is' model and its constraints and instead look into what should be and what could be."”
During the discovery phase, the process of capturing requirements begins, typically in the form of user stories. The team shouldn't overlook nonfunctional requirements, including those related to DevOps. For example, you might have a nonfunctional requirement outlining the requirement for a continuous deployment capability, which drives the development of scripts to support that requirement.
Including a DevOps approach in capturing requirements requires IBM BPM analysts and architects to envision beyond the "as-is" model and its constraints and instead look into what should be and what could be. Ultimately, the team envisions a "to-be" model.
Requirements are distilled into user stories. The team maps user stories to the process based on how the stories relate to the actor, their action, their task, or their expected benefit. The discovery effort culminates with a Playback Zero presentation to bring all parties together to understand process models, value, and scope.
Depending on the complexity and size of the project and the teams involved, the discovery phase can be linked with a macro design phase that focuses on vetting risk areas. For example, your team can vet risks related to architecture, service integrations, and user experience design patterns. You estimate, group, and plan for iterative delivery of the user stories that you previously defined.
The development phase
An IBM BPM development project should follow agile practices and principles. Key concepts include working closely with the customer on each feature, managing technical debt, and keeping overhead to a minimum. During a typical IBM BPM project, the developers work in the Process Designer authoring environment, against the current version of the application, contained in the tip (for example, the "head" revision). To implement continuous integration effectively, the team must adopt an automated practice of pulling snapshots (for example, the "label" revision) and installing or deploying those snapshots to the Process Server.
Continuous testing is another matter to consider during the development phase. Testing from the tip while development is going on can result in errors just because a component is in the middle of being developed. A common pattern around this problem is to cut a snapshot at a known time that can be pushed to a Process Server environment for development testing. As previously described, the DevOps approach calls for an automated way to pull the snapshot in for continuous testing. Automated testing also can be a challenge, at least from the user interface perspective. As a coach (an IBM BPM user interface) evolves, the automated test scripts also must be updated. For other parts of the solution, such as integrations, consider using test harnesses, which can be written and reused. Test harnesses can be used for the inputs and outputs of human services that contain coaches.
Another agile principle is to deliver fast. Continuous integration and continuous testing provide the capability to deploy continuously to production. However, this is very often too much change for an organization to be willing to accept. A more regular cadence of bi-weekly or monthly can be established. However, even this approach can be a challenge in some organizations.
“"Some organizations are still of the mindset that a group of features, which take many months to develop, need to be released together because of constraints such as user acceptance testing or training. If you find your team in this situation, develop a release cadence that is time-based instead of purely functionality-based."”
From a user's perspective, some teams might decide that the amount of work that can be completed in a month or less is not enough functionality to justify a production release. Some organizations are still of the mindset that a group of features, which take many months to develop, need to be released together because of constraints such as user acceptance testing or training. If you find your team in this situation, develop a release cadence that is time-based instead of purely functionality-based. Most customers prefer a release or a "go-live" cadence of every 4-6 months.
Some organizations don't start user acceptance testing until after development is complete. Instead of taking this approach, get end users involved early. The business team and subject matter experts should be running playback presentations at the end of each iteration, not only to demonstrate what has been developed under their guidance, but also to gain acceptance from actual end users for the user stories that are delivered. The playback also acts as an introduction to testing, where business, development and testing teams are on the same page about what stories are ready to be tested at the end of an iteration. You don't have to wait until all development is completed before testing. More extensive testing can happen after critical mass of a process has been delivered. To make this effort effective and visible, use a tool such as IBM Rational Functional Tester so the team can see the status of each user story and mark user stories as accepted when they pass verification.
From a technical perspective, the work must be broken into small components. This includes everything from the business process definition to coaches, toolkits, and integration components. Some of these challenges can require a first release that takes more time and includes more functionality, followed with a continuous approach, as described in the following section.
Imagine that the development process is one continuous flow. The process owner has some user stories in priority order. The developers pull the next, highest priority story from the list, and working with the process owner and subject matter experts, develop the user story. They demonstrate the story to the process owner, who then accepts it. The process owner performs testing, and now the story is ready to go to production. If your process application is very modular in nature, this technique can work with small self-managed teams of representatives from business, development, testing and operations departments.
The deployment and operations phases
You can use an automated deployment capability of IBM BPM to create a snapshot and push it to a runtime server. From the Process Center console, snapshots can be automatically sent to any online servers, from a test environment to production. However, this is not the only option for deployments. When you don't have a direct connection between a Process Center and a Process Server, offline deployment is an option. Consider offline deployment if you want to restrict who can deploy to production or if you want to support a DevOps approach in a more complex environment.
So far, you learned to look at a DevOps approach from a pure IBM BPM standpoint, but what if your solution is more complex? For example, you might be using IBM BPM in a "headless" fashion in conjunction with a mobile interface, configuring for a cloud solution, or delivering a complex solution with multiple, related pieces of code where the code for all components has to be coordinated and deployed together.
For more sophisticated solutions, consider a tool such as IBM UrbanCode Deploy to solve problems. Each component, including IBM BPM, can be configured as a component of one overall application. In the case of IBM BPM, the offline server deployment mode can be used for all environments, not just production. For example, when it's time for user acceptance testing, all components can be pulled into the user acceptance test environment at the same time, whether IBM BPM, a mobile front end, or any other code. Continuous deployment becomes a possibility with IBM Urban Code Deploy, because all components are managed together.
The feedback loop
After a process application is in production, it's time to monitor the application. From a business perspective, IBM BPM can be set up to capture important measurements about the process. During development, the team should have set up tracking points that tie to key business goals. For example, if a goal is to make decisions within a certain timeframe, tracking points can determine how often that goal is met.
Monitoring is not just about business goals. Consider monitoring the health of the application. Tracking data such as CPU utilization or memory usage can identify problems that need to be addressed. The operations team needs to work with the developers during development to identify monitoring requirements. This is another area where nonfunctional requirements come into play. For example, imagine a nonfunctional requirement that states a database query needs to return results within 2 seconds. The ability to track this SLA must be built, and the operations team must ensure compliance. If an issue does arise, operations can work with the developers on the resolution.
If there is a complex set of monitoring requirements, the developers and operations team can work on a dashboard to provide the necessary visualization.
This tutorial walked through some of the concepts of the DevOps approach and discussed how you can apply those concepts in an IBM BPM project. IBM BPM can easily implement DevOps capabilities, including continuous integration and automated deployment.
When a solution involves more than IBM BPM, the solution becomes more complicated and you should consider implementing a more sophisticated approach to DevOps with tools such as IBM UrbanCode Deploy.
Now you are equipped to plan how to apply the DevOps approach to your IBM BPM projects.
The author would like to thank Jerome Boyer and Guy Lipof for their review and input to this article.
- IBM Business Process Management products
- IBM Business Process Manager product page
- IBM UrbanCode developer center
- IBM developerWorks Middleware
- IBM Redbooks about IBM Business Process Manager