Modified by Eric Herness
For the past couple of months, there has been a Bluemix application out there that should be of interest to all IBM BPM customers. Go to http://wombat.mybluemix.net/ and take a look.
Wombat consumes authored artifacts and does a static analysis of those authored artifacts. Specifically, it consumes the following formats:
- IBM Business Process Manager (all editions) v7.5 and newer: TWX and Project Interchange Files
- IBM Business Monitor, all versions: TWX and Project Interchange Files
- IBM WebSphere Process Server, all versions: Project Interchange Files
- IBM WebSphere Enterprise Service Bus, all versions: Project Interchange Files
- IBM WebSphere Lombardi Edition, all versions: TWX File
- Teamworks v7.0: TWX File
While we say this is of interest to IBM BPM, you can see from the list that it covered related and historical products as well.
When you submit your artifacts and wait a bit, you get an analysis report. This report consists of 3 major sections:
- Statistics and Archive Warnings – This is two sets of useful information packed into a single heading.
- Archiving Warnings – These are more specific. These provide a summary of possible concerns in the overall application. Some of these will be obvious things to fix while others should at least cause some deeper inspection. There are about 70 rules that are used to arrive at these warnings. Where available there are links to ways to address the documented concern and get more information. These warnings have been developed over many years (this tool lived on the IBM intranet for a long time before it’s coming out party in late 2015) based on looking at literally 1000’s of projects. These rules are also influenced by the lead technical engineers that work on these products and know what work well and what doesn’t on our runtimes. Remember, this is free advice and Wombat is objective, providing consistent review. That said, however, we do update the rules regularly, mostly by adding more, so feel free to come back and re-run as you wish.
- Detailed Report – This section gets long and involved. It has counts and warnings for individual artifacts such as BPDs, Services, toolkits, BPELs, coach views and the rest. This is what the developers charged with improving the application will need to get started on refining the solution. Toolkits are also listed here, with links to individual analysis on those toolkits.
When using this tool, your artifacts are safe. You will use your IBM ID to get to wombat. You will then upload your artifacts. From Your files are transferred to IBM’s server over secured and encrypted channels, are analyzed, and the results are returned to you over the same encrypted channel. Your result set is encrypted at the drive level.
Wombat has a community where you can provide feedback. There is a blog pointed to by the community as well. Please join the community to post questions or feature requests and will respond as soon as we are able.
That is the story of wombat for now. Go try it. If you are coming to InterConnect, feel free to seek out Ryan T Claussen and myself. Bring your results. We can answer questions or take suggestions real time.
Considering the Mobile Options for IBM Business Process Manager 8.5.6
What is your strategy for mobile as it relates to business processes? There are many elements to consider. A full strategy for leveraging mobile will look at the device as an opportunity to extend a process, gather contextual information and interact with people in new and exciting ways. That is a great topic to cover and one that is worth a number of deep dives.
However, if what is needed tactically is to jump start mobile enablement activities and to ensure that new things are built in a way that makes them mobile capable, then a few basic thoughts and some prescriptive guidance may be just what is needed. This short article will do just that.
To level-set the capabilities being introduced in BPM 8.5.6, let’s take a look at the new responsive portal, and a responsive coach launched from that new portal, running on an iPhone 6. Note that you could run this from your mobile device’s browser or you could deploy this portal to you device as a hybrid app:
With that view of responsive in mind, the following slide is intended to provoke thinking about mobile user interface investments around Smarter Process. Part of the motivation for this conversation is to solicit feedback and part of the motivation is to provide new ways to think about mobile enabling their processes.
As you can see, we support 3 ways to build out a mobile user-experience using the recently announced IBM Business Process Manager 8.5.6 product.
Web Application – Assemble write-once/run-anywhere coaches from responsive building blocks. This option has the lowest skill bar, and the best time-to-value, especially for B2E scenarios. BPM 8.5.6 introduces a new responsive coach views toolkit, built out of the latest UI technologies, that provide native-look-and-feel, across a variety of devices, including phones, tablets, and desktop browsers, all of which can be launched from the responsive portal running on such devices.
Native Application – These require more skill and time on the part of your developers, but are useful for “showcase” mobile apps that have very specialized user interface requirements. These are generally targeted at your customers, in B2C scenarios. Developed outside of BPM’s Process Designer and its governance model, these applications make REST services calls to BPM to start process instances, claim and complete tasks, etc. Of course, you have to develop a different native app for each platform you wish to support, using the SDK for your device’s operating system, such as one for Android devices and another for iOS devices.
If we step back for moment and look across a wide variety of processes that could benefit from mobile enablement, the assertion is that 80% of applications can handled, from a mobile perspective, by web browser based approaches. To make this work, in IBM BPM, the responsive coaches and client-side human services need to be leveraged during development. The development process can certainly include playbacks where mobile form factors are involved. Discussions about how well given tasks fit into the mindset and interaction styles prevalent on mobile devices can most certainly happen. The point is that by using our tried and true playback approach, combined with our responsive coach views, you can get a good mobile story, one that in the end is certainly better than more classic web applications built without any mobile first thinking. Note that the responsive coach views will have device specific rendering of various user interface elements because of the usage of HTML5 in the responsive coach controls.
Some applications, because of their nature, need to be downloaded and installed on mobile devices. This reduces initial latencies and allows for different security, management and monitoring. Since all of the UI artifacts (html, js, css, images, etc.) are part of the app and thus are local to the device, you don’t need to transmit these from the server, which is especially helpful on slower cellular connections. All that gets transmitted from the server is the values of process/task variables that should be initially shown on your coaches, and then the values that are entered by the user of the coach are transmitted back to the server when you complete your task. This approach also gives you the ability to publish your app to your corporate app store, and to push out updates to that app as needed.
As mentioned above, BPM 8.5.6 introduces the option to generate a hybrid application from your coaches. This gives you the ease-of-use of assembling your UI via drag-and-drop of simple building blocks in a web-based authoring environment, but still gives you the ability to produce mobile apps that can be installed to your devices, and centrally managed via your corporate app store.
This final category of applications includes those that require device specific precision and control over user interface interaction that exceeds that of our responsive coaches capabilities. To support this type of application development, IBM BPM 8.5.6 offers:
REST APIs – We have continued to evolve our REST APIs to meet the needs of native mobile application developers. BPM 8.5.6 also introduced new REST APIs for its new Process Federation Server, enabling you to get process and task data from a variety of BPM environments in a single REST call.
IBM MobileFirst Platform adapters – When using coaches is not an option, our preferred IDE for developing mobile applications is the IBM MobileFirst Platform Studio. We have a couple of approaches here:
Generic adapter – exposes each of BPM’s public REST APIs to the MobileFirst programming model. Rather than having to manually build up HTTP connections with appropriate headers, parameters, etc., you just invoke a procedure such as startProcess(processAppID, processID, inputParams)
App-Specific adapter – generate a strongly-typed MobileFirst adapter from a specific process app snapshot. This has procedures for each exposed process, task, and service in that process app, which ultimately call the corresponding method of the generic adapter. Rather than having to know and pass UUIDs and such, this makes it as easy as invoking the startHiringSampleProcess() procedure to start an instance of the Hiring Sample process, for example.
In the end, you must consider what provides the most value. Business process management is about managing work, making the business more efficient and providing visibility into what is working well and what is ripe for improvement. Processes mobile enabled via web applications can be created quickly and thus start delivering quickly on the promise of business process management. If the execution of a process and assessment of the effectiveness of that process suggests a stronger and more powerful user experience is necessary, then it is simple enough to begin to replace those web applications with hybrid or native solutions.
Building out mobile user interfaces for business processes is only part of a broader set of topics to consider when taking a mobile first approach to business process management. This article has focused on elaborating the different approaches that should be considered and has suggested some ways to very quickly build out mobile ready user experiences for a large share of business processes. More costly, but powerful approaches have also been elaborated upon and are appropriate in some suggested situations. IBM Business Process Manager enables all approaches to user experience enablement for mobile.
Thanks to John Alcorn, Smarter Process Mobile architect, for working with me on preparing this post.
Modified on by Eric Herness
Getting started with a BPM project or rolling out a BPM program each requires investment in setting up and configuring a set of BPM environments in which to develop, test, execute and improve upon those business processes which are being automated. There are many ways to tackle the challenge of getting the right environments ready to go.
The traditional approach to setting up environments for IBM Business Process Manager (IBM BPM) is to install and configure these environments on locally owned hardware or a locally managed virtualization platform. This usually means reading a redbook or acquiring the services skills and help to do the installation and configuration. Of course, that is preceded in some cases by acquiring the compute resources and getting them in house and running.
On the other end of the spectrum is the SaaS approach and in the world of IBM BPM, we call this IBM BPM on Cloud.
In the middle-ground between the traditional approach and the full SaaS approach we find the world of PaaS and for IBM BPM, this means patterns. Specifically this means patterns for PureApplication System or patterns that work in the context of Pure Application Service on Softlayer. More recently, we also support IBM Cloud Orchestrator with the IBM Business Process Manager Pattern 8.5.5.
This blog entry highlights some of the interesting and valuable things to know about our IBM BPM 8.5.5 pattern for PureApplication. Most of what is described here and in the related PDF also applies to Pure Application Service on Softlayer and the IBM Cloud Orchestrator variations as well.
The concept of patterns and the embracing of that concept by PureApplication System and our IBM BPM usage of the patterns framework available on PureApplication System provides for a powerful approach to setting up and configuring environments for IBM BPM 8.5.5. Powerful means:
that full environments can be setup in a few hours with a minimal set of inputs required.
that these environments can be customized to meet specific local requirements.
that these environments can be managed and evolved in a simplified and consistent way.
I will elaborate on a few of these topics in the following paragraphs.
The IBM BPM 8.5.5 pattern builds on the latest patterns engine from PureApplication System. This patterns engine delivers the best of both Virtual System Patterns and Virtual Application Patterns approaches found in earlier IBM BPM Patterns. Specifically, the new IBM BPM 8.5.5 pattern provides a high-level Virtual Application Pattern (VAP) as well as a number of default Virtual System Patterns (VSPs). The VAP is constructed from the VSP. The VSPs are constructed from various pattern elements. Customers may choose to start with the VAP, or may choose to start with one of the VSPs. If the VSP is customized, then a new VAP built on that customized VSP is simple to create. Typically, we expect that customers will start with the VSPs, customize them by adding script packages and add-ons to fit to their specific environments and then optionally repackage them into VAPs. It is also possible to start with the pattern elements and construct a VSP. This will not be common but is still a path customers can utilize.
The VSPs come setup for high-availbility and are ready for production usage. Some specific highlights of the 8.5.5 patterns include:
for those using the DB2 embedded in the pattern we have:
created the appropriate tablespaces for both BPMN and BPEL processes
configured for a single database rather than the usual 3 databases required by the classic on-premise version of IBM BPM
provided for DB2 HADR configuration for both database high-availabilty and disaster recovery
access, by using the DBaaS pattern, to the latest DB2, which we have tuned for IBM BPM
a set of the latest required and recommended i-fixes for IBM BPM
a set of the latest fixes for both IHS and the underlying WAS.
IHS tuning specific to IBM BPM
specific patterns for using external databases
specific patterns to facilitate various product migrations
This is just a sampling of the tuning and optmizations available in the patterns. There are many optimizations and out of the box tuning that are unique to the pattern. When we know the topology and the underlying environment, these optimizations make good sense and are intended to support production workloads. Of course, every workload will vary and additional testing and tuning are always required, but this is the best starting point available. We have truly embraced the 'expertise in the box' idea which is foundational to patterns.
A key challenge in deploying production BPM workloads is always that of getting the right level of high-availability and disaster recovery support configured. PureApplication System now supports the concept of multi-rack deployment. The IBM BPM 8.5.5 pattern supports multi-rack deployment, meaning a single pattern instance that spans multiple PureApplication System racks. All of the DR approaches supported in the standard on-premise product configuration are now available with the patterns on PureApplication System.
While there are many different patterns supported and many different ways to customize the patterns, the more workload you commit to the PureApplication System, the more value you will get. Use the DB2 that is available in the pattern and let the auto-scaling work for both the WebSphere custom nodes and the database nodes automatically adjust the number of CPUs, the amount of memory or the number of VM nodes. Run testing and production, probably in separate cloud groups. Run multiple process applications using multiple pattern instances to get great isolation and to begin to see the power of the technology of PureApplication System work for you.
While it isn't possible to elaborate upon all of the capabilities and features available in this latest version of the IBM BPM pattern, it should be clear from this summary that there is truly a lot of power here. Environments can now be consistently and quickly be created for development or production. Just think of being able to create an environment for stress testing that matched exactly what you had in production and do that in a few hours. Just think of being able to add more production environments in a few hours. A quick demonstration of this pattern is available here: https://www.youtube.com/watch?v=JnzbbihcAbU.
I am certainly excited about these patterns, not only because they are technically powerful and very customizable, but because of the value they bring to the broader cause of rolling out BPM in an organization. As business sponsors and champions of process automation and continuous process improvement, you no longer need to wait for infrastructure and debate alternatives.
Thanks to Ryan Claussen for preparing the pdf and for reviewing this content.
Modified on by Eric Herness
Business Process Change – Third Edition
Paul Harmon’s 3rd edition of this longtime must read in the world of BPM contains many of the essential messages of prior editions with some valuable new twists. Overall, I found this to be an easy and interesting read. That view of course, is one from someone that is engaged in the world of BPM and BPMS on a daily basis.
Some highlights, some comments and some new things I started to think about during and after reading this include:
Are ships often passing each other in the night as organizations work to adopt BPM? The book outlines business process change opportunities using three levels: Strategy or enterprise level, process level and implementation level. I suspect that in many organizations discussions of BPM and the usage of BPMS are actually ineffective because different constituencies are just talking past each other. The higher level executives are perhaps thinking about strategy and enterprise level topics and just do not see BPM as valuable there, because the implementation and process level managers are focused on automation and individual process activities. These lower level managers also often lack the experience and knowledge to link these levels of process management to those higher strategy and enterprise conversations.
The levels of process awareness and process interest and the need for alignment between these levels certainly gets one to think about our current BPMS capabilities for visibility. I would suggest that we might need to continue to refine and ensure that our business monitoring are tailored to and sufficient to cover all the roles in a business that are interested and needing visibility into the processes.
I certainly enjoyed the practical perspective on lean and six-sigma. These are valuable approaches, but only if they are part of a larger effort that includes BPM and ultimately the use of a BPMS. That might be a bit of an interpretation on my part, but that is certainly how I see it. Documentation alone just won’t get the sustainable and ongoing results that are being demanded by business today.
The piece that talked about departments that optimize their part of a process at the expense of others and slow down the entire end-to-end process certainly resonated with me. This happens all too often from my experience and is something to keep an eye on when working with individual departments on BPM activities. Clear guidance is provided on how to overcome this phenomena or avoid it in the first place.
Business process architecture is another of the topics that resonates with me and that I find is either missing or lacking in many large organizations. This leaves first projects to bear the burden of figuring out these things in addition to delivering the process automation project that they are likely funded to deliver. Gathering and retaining the right skills to do good business process architecture also seems to be a challenge. The right mixture of IT and business is essential to producing and maintaining a useful and meaningful business process architecture.
I have to say that the repetitively made points around the importance of linking BPM to business value chains is also a welcome message that I think everyone engaged in BPM needs to keep in mind. Periocically, when I speak to clients about the linkage of the current project to value chains, I get the blank stare. That is when I know the challenge at hand is great. Sometimes, however, there are good answers to these questions and in the end these come from the projects that are likely to be the most successful. Having that business level sponsor is a vague statement we make in the world of BPM, but an important one. Having the value chain discussion takes it to a next and more measurable level. BPM is, of course, here to support the business strategy. Later in the book there is a great section talking about subordinating “standardization” and “operational effectiveness” to the improvement of the value chain. That brings the whole topic into more clear focus.
The measurements discussion was a good remind for me of how important measures are and the influence these can have on behaviors. It also gets me to think about process taxonomies and how the measures may change or need to be somewhat different based on whether a process is much more strategic in nature versus operational. The financial element, of the balanced scorecard approach, for example may not warrant nearly as much weighting in processes that are exploring and nurturing innovation as it would for the processes managing basic goods and services production.
The ERP and BPM discussion and positioning I think is very good. This notion of fitting a business model to the constraints offered by an ERP can initially be a good thing and it can be limiting, especially over time. The idea that it will be difficult to create serious competitive advantage by merely embracing a large ERP is right on, in my experience, and was explained nicely. There is ample room for both BPMS and ERP in many user organizations. Whether that makes sense for a given organization will get back to value chains and business priorities. Finally, the idea that many of the challenges faced by those implementing ERP are the same as the ones found by those implementing BPM also resonates with what I encounter. Some of the same cultural challenges appear in both situations.
Near the end of the book is a passage I really liked that talked about the BPMS market. The notion that the market will expand is certainly good news to those of us in the business. However, I think the comment suggesting that the market is “largely gated by the BPM maturity of user organizations” is right on. I often tell customers of our IBM Smarter Process portfolio that we will keep evolving the technology, but their assignment is to prepare and ready their organization to embrace the overall BPM approach. We all have responsibilities here as we work to improve business through BPM.
Overall, this was a good read and should be something that anyone practicing BPM puts into their library and consults often. There is good balance throughout the book. This balance includes a lot of BPM and an appropriate dose of BPMS. While making the value of BPM truly persistent and ongoing requires a BPMS (and a BRMS and some other technical elements), it is also unquestionably the case that the commitment from the business to embrace BPM is essential. Finally, the injection of case studies and examples throughout the books also adds greatly to the understandability and overall flow.
Thanks again to Paul Harmon for sharing his experiences in this format.
Modified on by Eric Herness
Process Change: Plan for It, Expect It and Look for It
Part of embracing practicing BPM is the notion of continuous process improvement. This means processes change. In fact, I would contend that if your processes being executed in a BPMS are not changing at all, it is not generally a good sign. This blog post will share some thoughts about process change.
Process Change Occurs
Our experience is that indeed processes do change over time. The idea of automating a process before it has been fully modeled is a common and valuable one that is played out all the time in the industry. I like say to say that sometimes it is good to automate what you have and then “let the process talk to you” through the execution of instances against the process definition that is available. Avoid analysis paralysis and get started using BPM as soon as possible. That is often the message and this is a good message.
In various conference presentations over the years, these next two figures have represented a view of the fact that processes can change and in fact that you can start process automation at any level and then let the process do the talking on the journey to refinement.
Figure 1 - Process Shape Evolution
Figure 1 shows that a process that presents itself as human-centric in the early automations may indeed become much more straight-through in nature as time goes on. The constant pursuit of understanding and automating more of the work that the humans are doing, as well as removing duplication and rework, changes the process. The main path evolves to become more straight-through (no humans) while the human involvement becomes centered on using their expertise to handle the exceptions and complex activities. Ongoing process improvements should work to improve the exception handling and handle more and more exceptions in an automated fashion and then start to provide more valuable assistance to the humans working on the remaining exceptions. Much more could be said here, but the idea is that the process evolves.
I have called this shape evolution to highlight that while the intent of the process and the business objectives fulfilled many remain the same at the high level, the process itself does change and in fact leverages different features of an underlying BPMS.
Figure 2 is an older figure that shows that the number of system interactions changes as the process matures. The light blue activities are human tasks while green represents read-only system activities and red represents updates to systems of record.
Figure 2 - Top Down Process Evolution
While it does not completely reflect the straight-through nature of the process in the later versions, it does show evolution. Again, the human centric process is not really a human centric process, it is a business process first and foremost and evolution allows it to be more efficient and more valuable over time by interacting with backend systems and other IT resources.
With this perspective in mind, it is then interesting to point out that processes that might appear to be less critical at one point in time and at one level of understanding (a more limited understanding) might indeed become much more important later on as understanding of that process improves. I have used the graphic in figure 3, sometimes called the long-tail slide, to suggest a possible individual process moving from right to left.
Figure 3 - Process Important and Complexity
The net effect of process refinement and increased visibility may indeed change the business view of how important a given process is. If it is truly integrating and driving more and more systems of record over time, then it is certainly more complex and likely to be more critical to ongoing business operations than previously realized.
In summary, the only constant in BPM is change. There are two key things that have been described and offered as something to think about with respect to process change.
A process that appears human-centric today can easily become straight-through or nearly straight-through in the future. In fact, a process may need to do this in order to provide maximum business value in light of changing business strategies and market conditions.
A process that appears relatively simple and less than mission critical in early automations can become much more important and complex over time through the use of the BPM approach.
Look for future blog posts that build on this and explore additional topics related to automating and evolving business processes.
Some Updates and Clarifications on Autotracking and PDW
On occasion the Smarter Process CTO Office receives requests from IBM Support to help resolve customer issues and to perform BPM process application analysis. After much PMR work and application analyses over the past few years we’ve developed the best practices of 1) disabling auto-tracking on individual business process diagrams (BPDs) and 2) disabling PDW on production Process Servers if PDW information is not needed. This blog post will explain our reasoning, the impact disabling auto-tracking and PDW has on your production environment, and some alternatives available in the Smarter Process portfolio.
In select versions and editions of BPM (IBM WebSphere Lombardi Edition (WLE), Teamworks, BPM Standard, and the BPMN portion of BPM Advanced) autotracking and PDW perform a few main functions. First, auto-tracking enables a generic set of features which copies process, task, and performance information into the PDW database. This database manages historical data for all BPDs and tasks as they execute in a Process Server or Process Center environment. In turn this data is used for:
The Optimizer perspective in Process Designer which allows you to perform historical and as-is/to-be analysis against true historical data,
KPI and SLA information in the Process Portal UI,
Auditing history such as which business data was changed and when,
Heritage ad-hoc reports (deprecated)
Heritage Process Performance scoreboard (deprecated in 8.5)
BPD instance diagram path annotations, based on historical typical paths, in the BPM 8.5.x Process Performance instance dashboard. The instance path taken so far is marked by the blue lines while the projected path is marked by the orange lines in the diagram below.
As you can see, a few of these items have been deprecated in more recent IBM BPM releases (ad-hoc reports and heritage scoreboards) and will be removed in future releases. Other items, such as the Optimizer, are not heavily leveraged during production in client deployments my team and I have interacted with. If your BPM project is not utilizing the items in the above list you can safely unselect Auto-tracking on your BPDs by deselecting “Enable Autotracking” on the Tracking tab of a given BPD as shown in the image below. If you are authoring new BPDs in IBM BPM 8.5.5 or later you will find that autotracking is not automatically selected and no action is required to disable autotracking.
As of BPM 8.5, the Process Performance dashboard does not require auto-tracking to be enabled. Without auto-tracking enabled, the process diagram on the Process Performance instance dashboard will simply not annotate the traversed path, but will continue to display a projected path (and projected future tasks on the instance gantt diagram) based on the longest calculated future path through the process, instead of based on historical typical path. If end users have permission, they can modify the instance’s projected path.
If you want to track the performance of BPDs we prefer you add Intermediate Tracking events and tracking groups. These constructs allow you to explicitly define which pieces of process data you want to capture at a given step of the process. If you define custom timing intervals based on the Intermediate Tracking events, an additional tab will get added to the Process Performance dashboard to display average timing interval durations. We have a developerWorks article which provides an overview of Tracking Groups along with the regular information provided in the IBM Knowledge Center. Please see the references for the links to these resources.
If no BPDs will utilize auto-tracking or Intermediate Tracking events, you can completely disable the PDW by following the steps from the technote referenced below.
If you are using the Performance Data Warehouse information for auditing it is important to adopt an archiving and purging strategy for data management. We’ve found that as the PDWDB grows the general BPM system performance begins to slow down. There is no system provided PDWDB archiving support in BPM today so any archiving solution will be roll-your-own. An option is to ETL the required PDW information into some other data warehouse solution such as IBM DB2, IBM Cognos, or IBM PureData appliance. IBM BPM does provide purging support for PDW as outlined in the references at the end of this post.
IBM does provide a complementary product, IBM Business Monitor, which integrates with IBM BPM and provides many of the same features and functions as the Performance Data Warehouse. With Business Monitor you can generate KPIs, visualizations, reports, and custom dashboards for IBM BPM process applications. In IBM BPM and Business Monitor 8.5.5 we’ve provided quicker and easier ways to utilize Business Monitor earlier in the development lifecycle via automatic generation and deployment of monitor models to Business Monitor, all from Process Designer.
Thanks to Ryan Claussen and David Enyeart for assisting in crafting and validating some of the materials in this post.
Key References for this topic
Steps to disable PDW on PC and PS: http://www-01.ibm.com/support/docview.wss?uid=swg21611013
PDW and its impact on performance dashboards In BPM 8.5.x: http://pic.dhe.ibm.com/infocenter/dmndhelp/v8r5m0/index.jsp?topic=%2Fcom.ibm.wbpm.wle.editor.doc%2Fmodeling%2Ftopic%2Fcritical_path_A.html
Purging in IBM BPM, updated July 2014: http://www.ibm.com/developerworks/bpm/bpmjournal/1312_spriet/1312_spriet.html
Creating Monitor Models in Process Designer: http://www-01.ibm.com/support/knowledgecenter/SSFPJS_8.5.5/com.ibm.wbpm.mon.doc/scen/bpm.html
Monitoring business processes by using tracking groups in IBM BPM v7.5: http://www.ibm.com/developerworks/websphere/library/techarticles/1108_venugopal/1108_venugopal.html
Tracking IBM Business Process Manager performance data: http://www-01.ibm.com/support/knowledgecenter/SSFPJS_8.5.5/com.ibm.wbpm.wle.editor.doc/topics/how_perfsvr_stores_data_a.html?lang=en
Customers often ask me why they need BAM, in the case of the IBM Smarter Process Portfolio, why do they need IBM Business Monitor. The answer is a bit complex, but offers a key lesson about business process management in general. This blog post will begin to tell this story and leave some room for interpretation and discussion.
The net is that you do need IBM Business Monitor to get seriously started with BPM. The question is to figure out not ‘if’, but exactly ‘when’ you will need it. At least that is my view. The short answer is you need it sooner than later.
Before proceeding down the prescribed path, we should make some observations about areas to watch out for related to business activity monitoring. First, business activity monitoring is not IT monitoring, nor is it a substitute for good IT monitoring. Secondly, business monitoring is your ally and your friend. It can reduce the costs incurred under the heading of ‘custom reports’. I see lots of time and money spent on custom reports that can be largely covered by the capabilities of IBM Business Monitor.
Now we should get back to the main story. For processes that are implemented using a top-down approach, one that encourages early deployment in favor of getting caught in analysis paralysis, we should turn our attention to continuous process improvement.
It is often the case that very elaborate dashboards and KPIs are not needed on the first day a process is automated. In the case of IBM BPM, it is often sufficient to look at the built in capability of the process performance dashboards, team performance dashboards and other built in reporting that come with IBM BPM. These capabilities, even in the out of the box form having not been customized, should quickly highlight areas to improve. For example, the process performance dashboard shows where time is being spent and the team performance dashboards show key bottlenecks in staffing from either a capacity or skills perspective. Several iterations of process improvement should come easily from this information, each using a little more of the insight that IBM BPM has to offer.
At some point though, when the obvious improvements to the process have been made, a transition begins to occur. This means two things. Team leaders and business users responsible for a process and its execution from day to day will stay relatively comfortable with built in dashboards. Business process analysts and those looking for further process improvements however will feel as though there is little left to learn from the available reporting and information. They will have essentially ‘used up’ what is there to leverage. They will want more.
For those looking for further process improvements, it is time for a real BAM capability like IBM Business Monitor. IBM Business Monitor will allow more in depth analysis of processes in flight and completed. Business process statistics can be joined with relevant business data flowing through the process to provide additional insight. For example, you can know and need to know that not only does a given task take too long on average, you know that only some of the tasks are outliers and it is the region, or loan amount (or other relevant business object information) that discriminates how long it is taking to perform a task. An improvement action can now focus on giving task workers special data or even special subtasks that improve overall process efficiency.
Process improvement can feed off this new found insight for a long time. Process refinements will lead to more specialized KPIs (Key performance indicators), use of predictive capabilities and the ability to make faster changes and take actions more quickly when deviations arise. The built in Cognos capabilities of IBM Business Monitor are essential to getting those ongoing improvements.
With this ongoing focus, these human-centric processes often begin to have straight-through paths. Rules replace humans for simple decisions. Life is getting better. The value of BPM is paying off.
The improvements possible with the various features of IBM Business Monitor are substantial.
Process improvement continues, but soon begins to show appetite for even more insight and information. Leveraging events from other sources, such as services invoked from the process, is something that is easily accomplished with IBM Business Monitor. This is a key contributor to the overall value gained by using IBM Business Monitor.
Beyond this point of process evolution, various analytical approaches, including using scoring models and deeper use of business intelligence tools will fuel further process improvement. For example, we have certainly seen cases where processes are changed to consult, via a web services call, scoring models at strategic points in the process. Sometimes the results of these inquiries are used to make decisions and in other cases they are offered to human working on assigned tasks requiring judgment and insight.
Another view on when to inject BAM and IBM Business Monitor into the BPM project rollout would suggest that it should be in place immediately. That is fine too. This might offer up insight on more lucrative process improvements versus the approach I outlined of incrementally working through the various types of insights that are available. There’s nothing wrong with that at all. For processes that are very refined before they go live for the first time, this in fact might be the preferred approach. Another sidebar is that if you are getting infrastructure and operations setup for your BPM projects, it might be more cost effective to have IBM Business Monitor installed and configured at the same time.
So in the end, there are a couple of ways to leverage the Smarter Process platform. The platform can be an advanced application development platform or it can truly be the enabler for continuous process improvement. If you are not using IBM Business Monitor or are wondering why you might need it, then you might not be truly focused on continuous process improvement.
If you are using IBM Business Monitor or are going to be soon, make sure to check out the latest release, 184.108.40.206 which is announced here: http://www-01.ibm.com/support/docview.wss?uid=swg21671372
Modified on by Eric Herness
When IBM BPM was announced in 2011 with Version 220.127.116.11 one of the key value propositions centered on the combined capabilities of both business process diagrams (BPDs) based on BPMN (Business Process Modeling and Notation) and business processes described in BPEL (Business Process Execution Language). Advanced Integration Services (AIS) provided a high fidelity bridge for getting from BPDs to BPELs as well as a way to get at other powerful integration capabilities that come with the full IBM BPM Advanced offering. Since those early days of IBM BPM patterns of use and best practice have emerged related to this topic of AISes.
Technically, AISes can consist of almost anything that can be authored using IBM Integration Designer. This includes BPELs, MFCs (mediation flow components), POJOs (plain old Java objects), and adapters. These are authored and tested as SCA components in IBM BPM in the same manner as the business and integration components were back in the WPS (WebSphere Process Server) and WID (WebSphere Integration Developer) days. Packaging guidance for these components when they are accessed from Process Designer-created BPMN processes is provided below. There are essentially three packaging approaches to be considered.
AIS Façade 1 – This is where the advanced integration service is packaged in a process application that is separate from the application calling the AIS with a façade toolkit between the two process applications. See Figure 2 for an illustration.
AIS Façade 2 – This is where the advanced integration service is packaged in a classic EAR file.
AIS Embedded – This is where the advanced integration service is embedded in the primary process application.
Each of these packaging options each has their own pros and cons which must be considered. AIS Façade 1 allows for good separation of business and integration logic while also providing separate life-cycle management of the various parts of the solution, all within a single repository which is that of the Process Center. However, the addition of another process application will generate more process applications in your Process Center than the basic AIS Embedded approach. AIS Façade 2 provides separation and offers a way for AIS developers to leverage the more classic, WPS-style, EAR approach to packaging. This approach reduces the number process applications snapshotted and managed by the Process Center but creates a different set of artifacts which must be life-cycled and managed outside the Process Center repository. Deciding which approach to use may also be influenced by the type of advanced integration service being created and the rate and pace at which those services will evolve. The following figure offers an approach for selecting the right AIS packaging model and pulls together this topic in general:
Figure 1 - Decision Tree for Advanced Integration Services
Starting from the top left the first question is around whether or not there is already an SOA or existing set of services inside the enterprise. If so, these services might be available via an ESB but perhaps were not constructed using a schema that matches what is appropriate for the business process realm and thus some mediation or transformation may be needed. If an AIS is needed then one of the façade patterns outlined above should be used. In this case the pattern used, either AIS Façade 1 or AIS Façade 2, will depend on the nature of the team providing the AIS and how these services will be managed.
The other branch from the starting point tackles advanced integration services that will not live on the corporate ESB or come from other existing assets. The first question here is aimed at figuring out the development team and life-cycle approach which will be leveraged. If the same folks are delivering both the services and the business processes then the AIS Façade 1 path for packaging is appropriate and service implementations can be constructed with any of the editors that IBM Integration Designer offers up for capturing business logic. If there are different teams or organizations doing new integration services that support a BPM program and these services will live in a corporate SOA layer, then another set of decisions is needed. If those service interfaces are built bottom-up in a technical way then a facade will be needed inside the BPM program, as discussed earlier. These services could even be web services implemented in java that run in another instance of IBM BPM Advanced. If the services being built are customized to run in IBM BPM Advanced and have WSDL interfaces compatible with the current XML schema support in IBM BPM, then AIS Façade 2 can work nicely as well.
The summary is that there are two AIS Façade patterns recommended for use, depending on the situation being presented. Seldom would the AIS embedded pattern be leveraged.
Both of the AIS Façade patterns involve a version aware linkage between the calling process application and the AIS implementation. This is graphically shown for AIS Façade 1 in the following figure:
Figure 2- AIS Facade 1 detailed linkage
Additional information on AIS Façade 1 can be found at http://www.ibm.com/developerworks/websphere/bpmjournal/1106_taylor/1106_taylor.html and http://www.ibm.com/developerworks/websphere/bpmjournal/1112_pacholski/1112_pacholski.html .
It is also recommended that thorough unit testing of AIS implementations be done using the IBM Integration Designer test environment before publishing them to the Process Center via a process application. This reduces load and contention on the Process Center environment. Deployment times of process applications containing advanced integration services depend on the resources available to the Process Center’s deployment manager and can take a few minutes. This is yet another reason why having AISes separate from process applications going through rapid playback cycles (and frequent changes) is recommended.
This topic so far as focused on BPDs talking to services. In the case of a BPEL calling a BPD human-centric process a slightly different path is followed. In this case, the BPD and BPEL processes are always packaged in the same process application. Perhaps more on this subject is an appropriate topic for a future blog entry.
Hopefully this quick summary shows how our thinking has evolved around how to best drive service calls from BPDs in IBM BPM. Thanks to Ryan Claussen and David Booz for helping me with this blog entry.
Modified on by Eric Herness
The IBM Smarter Process team has recently released Fixpack 18.104.22.168 for IBM BPM. It is recommended that customers on 22.214.171.124 that are not migrating to 126.96.36.199 soon should apply this fixpack. This blog entry provides some clarifications and details about the state of this fixpack and hopefully removes any confusion that might exist.
First, customers should strongly consider updating to 188.8.131.52 rather than staying on the 7.5.x codeline. There are many functional and quality improvements in the 184.108.40.206 codeline. Remember that 7.5.1 was delivered in 2011, 8.0.x was 2012 and 8.5.0.x was a 2013 focus for the engineering team. One of the questions is whether customers should go directly from 220.127.116.11 to 18.104.22.168 or should they go to 22.214.171.124 first? If you are stable on 126.96.36.199, then go directly to 188.8.131.52 in terms of migration. If you need to get stable, it is possible using some of the new capabilities in 184.108.40.206 that are described below will be beneficial. Migration guidance always starts with ensuring what you are going to move is stable and predictable.
Next, as is the case with all fixpacks, there is some important background to understand. In order to ensure appropriate testing of all changes in a maintenance package, there is a point during development when no additional content is added to the maintenance package. This is normal process and procedure for fixpacks. As a result, there are a small number of fixes delivered just prior to the release of the maintenance package that are not included in the maintenance package. Some customers will be running fixes of this nature. These fixes are available on top of the new fixpack. During an update, IBM Install Manager will identify these fixes and display a warning with the list of unresolved APARs. Because these issues are not addressed in the maintenance package, fixes must be subsequently applied to prevent reoccurring problems. Below is an example of the message:
"WARNING: One or more fixes will be uninstalled when IBM® Business Process Manager Advanced is updated to 7.51.2. The update does not address issues that were resolved previously by the maintenance packages. The problems might return if fixes for the following issues are not reapplied or have new fixes applied to prevent the problems from returning."
In addition to the APAR fixes, the 220.127.116.11 maintenance package includes new serviceability and operational improvements:
New incremental update procedure: the fixpack can be applied to process servers first, starting with test, then staging and when fully tested then to production, and finally after that to process center, process designer and integration designer (do all last of these last three at the same time).
Improved performance of LSW_BPD_INSTANCE_DELETE stored procedure for deleting process instances.
New Performance Data Warehouse prune command for removing older data from the PDW
New BPMDeleteDurableMessages command to remove durable events
Improved Process Monitor page of the Process Administrator Console. Among other things by now including the time for services to run as part of the time rollup for processes. This data is also now accessible via an MBean.
Improved performance of the My Team Performance and My Performance scoreboards
Improved performance of process instance migration by using multiple threads. There is also a new 100custom.xml setting to configure instance migration to not migrate completed tasks, which can have further significant performance enhancements.
Summary of key links:
Updates will be made to this blog entry if there are any further clarifications required.
Modified on by Eric Herness
Projects to Programs – An Introduction and Initial View
One of the big challenges facing large companies today in the area of Smarter Process is to figure out how to fully capitalize on the BPM opportunity. Through encouragement and guidance, initial BPM projects are delivered to gain experience and skills in applying BPM. These are often contained within a single part of an organization. They cover one or a few key business processes. Through my travels and visits with customers as well as trying to keep up with papers and articles that touch this space, my views continue to evolve. I’m sharing a snapshot here for others to read and reflect upon.
Irrespective of how wildly successful initial projects are found to be, business stakeholders, most of whom are always on the lookout for opportunities to improve the bottom line, often find these early BPM projects. The business value delivered in these limited scope projects becomes the basis for proposals for enterprise wide adoption of BPM and what is often called a BPM Program. The project to program transition is the then next big challenge.
There is not a specific formula or approach that is guaranteed to work for those moving from initial BPM projects to an enterprise-wide BPM program. However, there are a few key things to consider and a few techniques that can help reduce risk and increase chances for success.
Linking BPM to the Business
The first part of setting up for a BPM program is to assess the readiness of an organization to deploy a full BPM program. While there are a number of ways to do this assessment, I prefer to engage with the owners, practitioners and stakeholders that are experienced with the initial set of projects, most of whom will be part of rolling out a full-scale program. In addition to those experienced with the early projects, additional stakeholders and participants that will play a key role should be engaged.
The business stakeholders need to be interviewed. While technical readiness is also important, I find it valuable to engage with the business stakeholders directly and in conjunction with those in the IT function that are responsible for that linkage.
The business stakeholders are the ones that will relate a business goal and vision. They will have a strong role in inventorying and deciding which processes should be automated. Getting agreement on high level objectives and concrete measurements that reflect progress against those objectives is essentially to rolling out a successful BPM program. These business stakeholders will be funding the BPM program, so they have to be on board and excited about the opportunity. In most cases I see, they need to initiate the movement from project to program, or at least be co-sponsoring the movement with the forward thinking IT folks that see the value proposition.
The redpaper at http://www.redbooks.ibm.com/abstracts/redp4898.html?Open has a strategy chapter in the beginning that further defines and outlines how to get the needed business sponsorship and to get the business objectives in place that can be used to measure progress.
The other people an organization has that can fill the various roles to be played in rolling out a BPM program must be also inventoried. Additional outside skills might need to be brought in to supplement an existing organization’s capacity. Outside skills should be used to deliver projects but also to mentor and train in house capacity. If an organization is implementing key processes that support their most important value chains, then to me, this seems like work that should be done with in house skills. Remember, these are not projects that are over in a few months. True BPM means continuous process improvement. Funded projects that terminate upon first delivery just don’t match the model.
With the business stakeholders on board and a view on skills and the organizational capacity at hand, it is then time to turn to the more technical elements and assess the assets and challenges in these areas.
Technical Infrastructure and Topology
The technical infrastructure required to support a BPM program will go well beyond that of supporting a few initial projects. First, a real platform goes beyond BPM, moving towards what I call a ‘Smarter Process Platform’. This platform includes not only IBM BPM, but also IBM BlueWork Live, IBM Business Monitor and IBM Operational Decision Manager for starters. Rolling out a BPM program without these key elements is possible, but this is the recommended breadth of capability to give due consideration.
The scale out model for the technical infrastructure also becomes bigger when thinking about rolling out a program. This means for example, rolling out multiple IBM BPM cells and having infrastructure to scale to more concurrent users, a large numbers of groups and lots of active process instances and human tasks at any one time. This also means looking at things like multiple Process Centers and Decision Centers. While the cloud and virtualization offer great accelerators when it comes to getting the infrastructure rolled out, the basic layout and capacity requirements are something to understand right away and plan for. Knowing a ‘Smarter Process’ program is going to be rolled out will cause different infrastructure decisions to be made than those that were in place to support a few initial projects.
Operations and Procedures
Operational procedures for managing and monitoring this platform also need to be planned for and accommodated in the overall approach. This is often something that is overlooked or underspecified in early projects. It is often possible to get by with less here for initial projects. Scaling out in this area, however, demands that the manual steps of before be automated and the reactive approaches to problem solving become proactive and documented.
How the platform is leveraged by the program participants must be considered next. I talk about this as application design or sometimes use the term “fit for purpose”. Processes that will evolve based on continuous process improvement from human dominated processes to processes that have a strong straight-through element must leverage many of the valuable features found in IBM BPM. Evolving processes can also mean having separately defined enterprise class decisions using IBM ODM. Features from other products available in the enterprise such as integration and analytics need to be blended in with the combination of capabilities from IBM BPM, IBM ODM and IBM Business Monitor.
These technical elements of infrastructure and application design are leveraged in a development process that features iteration, playback and other means of regular and deep interactions between the business and IT stakeholders involved in rolling out the BPM program.
Center of Excellence
Getting all this sewn together into an approach that works for a specific organization that can be managed does not just happen by having done groundwork on business linkages, laying out of the infrastructure, documenting operations and paying attention to application design. Something is needed that helps pull all of this together and provide end to end governance over the BPM program.
In my experience, getting that organizational capacity and continuous process improvement approach rolled out properly requires some sort of center of excellence (COE) or center of competency. My recommendation is that a COE does more than advise. A COE needs to provide participants for key projects that are part of the BPM program. They need to create reusable artifacts, and I mean code artifacts such as BPM Toolkits, that can become accelerators for individual projects and can become a basis for consistently solving many of the various things that come with rolling out a BPM program. Toolkits that have common user interface elements and service interfaces that do backend system integration are two examples of artifacts a COE should create.
The COE also becomes the champion and evangelist for continuous process improvement. They define and administer the approach for governance, iteration and playbacks. They essentially own and keep an eye on the process of continuous process improvement.
To summarize then, the elements to be considered and analyzed when rolling out a program are shown in this graphic.
The importance of a COE to pull together and provide continuity is highlighted once again. The COE needs to be at the heart of any successful program.
Rolling out a Smarter Process program is big task. There are many more details to consider than are laid out in this brief blog entry. Look for future entries to dive deeper into some of these topics.
Modified on by Eric Herness
Welcome to the Smarter Process CTO blog.
Expect variety in the blog entries found here. The day to day activities that I lead and those that I participate in are all over the map and the ones worth blogging will represent that diversity as well.
At the moment I have a number of things bubbling around for which there are candidate blog entries emerging that might be of interest. These might include:
1. IBM BPM releases we have in the field today, which ones should customers be using and how do they ensure they are getting the most from what they have.
2. Enterprise architecture and how Smarter Process, including processes, rules and events fit into an overall enterprise view
3. Smarter Process and how it relates to the hot topics of the day, namely cloud, big data, analytics and mobile.
4. BPM projects to programs - This is a very hot topic right now for me and a challenge facing many clients.
These items are just a few of the possible topics. As you can see topics will certainly go beyond just that of the products that are in the IBM Smarter Process portfolio.
I look forward to putting content out here and to the subsequent interactions that will take place. I hope it sparks ideas and provides some value for those who follow along and look at the content.
Smarter Process CTO
IBM Software Group