How to monitor your System z business processes

The purpose of this article is to determine which business processes can be monitored on System z™, and to highlight the steps necessary to create the event emitters that can then be consumed by the monitoring environment. While the procedures to do this are identified here, they are done so in a highly condensed fashion to provide this information as efficiently as possible. Resources point to additional informatioin where all of the steps in the process are elaborated upon. This article assumes some knowledge of IBM® WebSphere® Integration Developer and the IBM WebSphere Business Monitor Toolkit. This content is part of the IBM WebSphere Developer Technical Journal.

Share:

Rob Rowe (rprowe@us.ibm.com), System z BPM Marketing, IBM

Rob Rowe has been an IBMer for 11 years, and in that time has enjoyed four positions in the company, including software development and creating live demos using WebSphere products.



30 July 2008

Also available in Chinese

Introduction

A business process model consists of a number of business processes, choreographed to show the flow of the processes, which includes decisions, flows based on those decisions, human interactions, and more. Business process models typically begin with their creation in IBM® WebSphere® Business Modeler. Alternatively, they might have been illustrated in a Visio diagram, a FileNet® P8 diagram, or a diagram from another source that is then imported into WebSphere Business Modeler for changes, optimization, or continuance. In any case, the model is then exported from WebSphere Business Modeler in Business Process Execution Language (BPEL) to IBM WebSphere Integration Developer, where the service-oriented architecture (SOA) service assembly occurs, and then it is ultimately exported (again in BPEL) to WebSphere Process Server for runtime deployment.

Once the process model is deployed, you can monitor your System z™ business processes with IBM WebSphere Business Monitor, a comprehensive business activity monitoring product that provides an up-to-date view of your business performance. Personalized business dashboards process business events and data, and also calculate key performance indicators (KPIs) and metrics. WebSphere Business Monitor is built on the concept of event monitoring, which is the monitoring of system-generated events that represent changes in status. This type of monitoring is also known as business activity monitoring.

Business activity monitoring shortens the latency period between a significant business event and notification of the right person in the organization. That person can receive notifications in real time via e-mail, pager, and SMS messages. Upon critical notifications, that person can direct a business analyst to change the process flows appropriately based on the information received, and to react to marketplace events and unexpected situations. For example:

  • Industry verticals such as energy, utilities, and communications can use business activity monitoring to detect emergencies and supply & demand disparities.
  • Healthcare businesses can be alerted to HIPAA compliance errors, detections of illness outbreaks, or to find lost or contaminated blood.
  • Financial services can monitor for fraud detection by monitoring the geographies where a particular credit card number is being used.

Hopefully, these brief examples enable you to glimpse the potential benefits of business activity monitoring.

Get the most from this article

For this article to be completely understood, some knowledge of the WebSphere Integration Developer tool and the WebSphere Business Monitor Toolkit are required. This will aid in your understanding of how to create the common base events, create and configure the monitor model, and perform the unit testing. Knowledge of WebSphere Business Monitor will aid in your understanding of the monitor model deployment and dashboard personalization, as they are addressed in this article.


What can be monitored?

Monitoring Events from WebSphere MQ Workflow
A FlowMark® Definition Language (FDL) file from WebSphere MQ Workflow is used to generate event definitions and a monitor model. The monitor model is augmented and completed in the Monitoring Model Editor to process the generated events and calculate the metrics to monitor. This model is then exported to WebSphere Business Monitor for deployment.

You can natively monitor any business process running in WebSphere Process Server, WebSphere Appliication Server, WebSphere ESB, WebSphere Partner Gateway, FileNet and WebSphere Message Broker. These products natively send events without adapters or instrumenting code to emit Common Base Events. In addition, any application that generates events in the Common Event Infrastructure (CEI) can also be monitored. For applications which do not produce Common Base Events, such as SAP, IBM offers adapters to provide that functionality. For custom applications for which no adapter is available, you can create your own event emitters using the IBM WebSphere Business Monitor Development Toolkit.

Realize, though, that events must be enabled for WebSphere Business Monitor to catch them. More on that later.


The System z operating environment

The procedure for monitoring your business processes on System z is essentially the same as any other platform. The difference is that, as of this writing, WebSphere Business Monitor is not currently supported on z/OS® and therefore must be installed elsewhere, on another platform that is supported, such as z/Linux®.

As shown in Figure 1, your business processes are running in WebSphere Process Server on z/OS. The monitor server is running on a separate server and receives the events from the process server through a service integration bus (SIBus). As with the business process model, the monitor model is created in WebSphere Integration Developer and exported to the monitor server.

Figure 1. Typical System z configuration
Figure 1. Typical System z configuration

Common Event Infrastructure and Common Base Events

As mentioned earlier, WebSphere Business Monitor can monitor Common Base Events along with the Common Event Infrastructure. A definition of each is in order before we progress.

  • The Common Event Infrastructure (CEI) is IBM's implementation of a consistent approach for the creation, transmission, persistence, and distribution of a wide range of business, system, and network events, based on Common Base Events.
  • A Common Base Event is anything interesting that occurs from a business perspective that you wish to monitor. These events might include an excessive time delay in an ordering or delivery process, or perhaps an inventory issue on the part of one of your suppliers that requires an immediate action.

WebSphere Business Monitor architecture

WebSphere Business Monitor V6 enables a business client to monitor the runtime behavior of business processes through a Web application that is deployed on WebSphere Process Server V6 or WebSphere Application Server V6.1. The data that will be monitored will primarily be issued from a runtime engine. Data is encapsulated in Common Base Events by means of event emitters and is transmitted via the CEI.

WebSphere Business Monitor is responsible for its own data store for keeping the data required for the monitoring operation, including instances of running monitoring contexts and metric values. Analysis of data is made available by REST (REpresentational State Transfer) services, a standardized database interface that abstracts the database from the applications.

Monitoring of data is through the monitor model (Figure 2), which is a container that holds information about the business performance management aspects of a business model, including the business measures that are required for monitoring. Using WebSphere Business Monitor, you decide which processes to monitor, whether to monitor their subprocesses, what business measures to use, and then create a monitor model that you export to WebSphere Business Monitor. The model contains all the defined business measures (metrics, KPIs, counters, and stopwatches) in the process as well as its subprocesses.

The monitor dashboard (either Web-based or portal-based) displays KPIs, gauges, instances, dimensions, alerts, diagrams, and reports. Organizational views are available in the portal dashboard only.

Figure 2. Architecture diagram
Figure 2. Architecture diagram

WebSphere Business Monitor Toolkit

The WebSphere Business Monitor Development Toolkit is the central development tool for monitoring. The toolkit enhances the development experience by providing the Monitor Model Editor to assist you in creating monitor models. The WebSphere Business Monitor test environment can be installed with the toolkit into WebSphere Integration Developer to facilitate testing your monitor models prior to deployment.

Figure 3. Monitor model development process
Figure 3. Monitor model development process

As shown in Figure 3, the toolkit contains two components: the Monitor Model Editor, which is used to define all the elements of the monitor model (such as triggers, counters, stopwatches, measures, and KPIs), and the Monitor Server Unit Test Environment, which is a fully configured monitor server on which you can both run the business process application in the underlying WebSphere Process Server and monitor it at the same time. In this configuration, the events are captured directly from the CEI database without a need for an SIBus.

The output of the toolkit is a deployable monitor model application (an EAR file) that is installed in the monitor server. The monitor dashboard is configured to display the calculated measures in either Web or portal pages.

In a production environment, WebSphere Business Monitor runs on WebSphere Application Server, but in the WebSphere Integration Developer test environment, the monitor server runs on WebSphere Process Server, included with the toolkit. The dashboards in the toolkit are identical to those of a production environment.


Steps to monitoring

So far, we have touched on the benefits of business activity monitoring, the System z operating environment, the events and event transport mechanism, the monitor architecture, and the monitor toolkit. With this groundwork in place, let’s look at the steps involved in creating a monitor model you can use for monitoring your business processes running in WebSphere Process Server. The high level tasks are:

  1. Determine which processes you want to monitor
  2. Produce the Common Base Events for the monitor
  3. Create the monitor model
  4. Configure the monitor
  5. Perform unit testing
  6. Deploy the monitor model
  7. Personalize the dashboard
  8. Monitor!

Wait a minute: Why do you need to do all of this? Wasn’t it mentioned earlier that you can monitor any processes running natively in WebSphere Process Server?

Yes, and these events are certainly available, but as was also mentioned, these events must be enabled in order for WebSphere Process Server to emit them to CEI so that WebSphere Business Monitor can consume them. Can you imagine the performance hit you would suffer if all of your processes and all of their potential events were enabled by default? Event emission is therefore disabled by default, and so those events that business analysts and executives want to watch must be explicitly enabled.

Here is a closer look at each of the above steps:

  1. Determine which processes you want to monitor.

    Collaborate with your business analysts to determine the business measures and activities required for monitoring. If you know which processes you wish to monitor as you are creating the model, you can specify them in WebSphere Business Modeler. There, you can define simple measures and KPIs using the Business Measures view. Alternatively, the monitoring can be the last stage of business process management. If you have already deployed your model to WebSphere Process Server and now want to add the monitoring capability to your running processes, you can accomplish this using the WebSphere Business Monitor Development Toolkit, as mentioned earlier. This is the approach taken in this article, since the intention is to enable you to monitor your running processes on z/OS.

    In this step, you fully define the business measures (metrics) to be monitored and how they will be calculated. Metrics are used to collect information and are used in the calculation of KPIs. One result of this exercise is a high level understanding of the events your WebSphere Process Server must emit. At this level, you must understand at what point in the process an event will need to be emitted, and what business information the event will need to contain. In the WebSphere Business Monitor Development Toolkit, start by opening your BPEL model (exported from WebSphere Business Monitor) and look at the process diagram. Decide which processes are most important -- from a business perspective, not an IT perspective. These are the ones that you want to monitor.

  2. Produce the Common Base Events for the monitor.

    Before you create the monitor model, you must select the events that WebSphere Process Server will generate. WebSphere Business Monitor will use the generated events to monitor the process. Events need to be emitted for the decisions and invocation actions in the process. WebSphere Business Modeler decisions become BPEL links in WebSphere Integration Developer. You can use WebSphere Integration Developer to modify the application running in WebSphere Process Server to emit the events to be monitored by WebSphere Business Monitor.

    Using the information you defined in step 1 about the placement and content of events, you must enable the appropriate application components to emit the events with the necessary information. This does not require coding. These are the components you can enable for event emission to WebSphere Business Monitor:

    • BPEL process components
    • Human tasks
    • Business state machines
    • Business rules
    • Generic Service Component Architecture (SCA) components.

    Figure 4 shows the events that are selected to be emitted, indicated by the yellow flag icons. There are flags on each of the six activities; the flag in the upper left is for the flow, and the flag in the upper right is for the process. There are eight flags total.

    Figure 4. Events to be emitted
    Figure 4. Events to be emitted
  3. Create the monitor model using the Monitor Model Editor.

    WebSphere Business Monitor uses your monitor model to process and react to events in the CEI, produced from WebSphere Process Server or elsewhere. The monitor model contains the monitoring context instance creation, termination, and correlation information, based on the events generated from WebSphere Process Server. This information defines how to monitor the application in the monitor server.

    Specifically, the monitor model describes how to:

    • Gather information from events that will be used to calculate the required business measures; optionally, information can be stored in a database for historical analysis.
    • Group together events about the same monitored entity.
    • Structure the information; for example, to enable dimensional analysis.
    • Combine the information; for example, to identify trends.
    • Identify business situations in near real time and trigger resulting actions by sending out events.

    Here's a tip: You might decide to implement a portion of your monitor model, test your application and model, then add more elements to the model, and so on. It is sometimes helpful to develop and test these steps iteratively and let your monitor model and application evolve as you go, rather than take this task on fully all at once. Alternatively, you could create several models, one for each process that you wish to monitor, rather than use a single model.

    1. Choose what to monitor

      Drill down on the event sources to view the pre-defined templates that are available for monitoring the event sources you have chosen. There are several templates to choose from based on common metrics and KPIs. Select the Emitted Events tab and view the events that are available for monitoring for the chosen event source. These are extracted from the business process model that was created in WebSphere Business Modeler. Select the ones that you wish to monitor.

      Figure 5. Select what to monitor and an available template, which will become the monitor model
      Figure 5. Select what to monitor and an available template, which will become the monitor model
    2. Choose how to monitor

      Available choices are none, monitoring context, and event group:

      • A monitoring context definition defines all of the data that should be collected about an entity (such as a process, customer order, or the stock level of an item in a warehouse) as the system is running. Each of its runtime instances (monitoring context instances) use incoming events to monitor a particular (real or abstract) entity, such as a particular process execution, the state of a particular order, or the stock level of an item in a warehouse. They collect information that -- individually or in combined -- is useful for making business decisions. The information is extracted from the data carried by inbound events, and is held in metrics, counters, and stopwatches, which represent the business measures that a monitoring context collects.
      • Event groups are containers for inbound events that enable you to group related events together to provide structure in the model without using a monitoring context.

      Monitoring contexts introduce overhead in the form of keys, cubes, and so on, whereas event groups are simply containers that are purely visual constructs and not represented in the monitor model XML file.

    3. Preview your monitor model

      The display shows you what you will be monitoring. Go back and make changes, if necessary.

    4. Create the required metrics

      In the Metric Details pane, enter the desired characteristics you want your metrics to include. Select the appropriate trigger condition in the drop-down list. The trigger conditions generate the situation events that initiate the alerts.

    Make sure that the monitor server is started and synchronized. If not, right click on the server and select Start. Select your monitor model and then Generate Monitor J2EE Projects. In this code generation step, the monitor model (the actual .mm file) is examined and the code to support an actual implementation of the model is produced. The artifacts generated are packaged into an Enterprise Archive (EAR) file; you do not work directly with this EAR file until you explicitly export the EAR project to the file system for actual deployment into the production environment. When you add (or remove) projects to (or from) the server, the WebSphere Application Server deployment steps are followed to deploy the EAR file to the unit test environment. Select Finish. When completed, check the Problems tab for any issues that might need to be resolved.

    (The Monitor Toolkit can also be installed into WebSphere Application Developer for J2EE monitoring, but this topic is beyond the scope of this article.)

  4. Configure the monitor.

    Use WebSphere Business Monitor’s action services to configure alerts or to define actions to take in response to business situation events (for example, send an immediate SMS text message). Situation events are sent from the monitor model as outbound events, and the alerts will be visible in the Alert view on your dashboard.

    1. Configure the alerts

      In WebSphere Business Monitor, navigate to Monitor Action Services => Template Definitions to configure an alert and then create the notification for the alert. Next, add the binding from the situation event to the action type of the alert. Without the binding, the alert will not be consumed as expected. Ensure that the situation event name is the same name that is defined in the model; this is case-sensitive, and so it must be an exact match for the alert to be consumed upon the occurrence of the situational event.

    2. Create the dashboard

      Dashboards enable you to access the monitored information: the metrics and KPIs. Requirements for the dashboards were determined when you assessed the process to be monitored and defined requirements for the business measures for monitoring. Remember that you define dashboards to fit the business needs of the roles in your organization, rather than having one that is process- or IT-focused. Once the dashboards are deployed, users can further tailor them to meet their own personal needs from the WebSphere Business Monitor console.

      At this stage, you need to create the dashboards for unit testing in the next step to ensure that events are emitted, calculations are made, and results are displayed. When you are testing a monitor model within the Monitor Test Environment, use the Web-based dashboards, as they are identical to those in the deployed, production environment. A typical mix might include KPIs, alerts, human tasks, reports, dimensions, instances, and diagrams. You might need to update the application or monitor model after you build and view the business dashboards to fully satisfy the business monitoring requirements in production -- or you might just want to create a simple dashboard at this point for testing and defer the creation of the fully functional dashboard to the monitor server once the monitor model is deployed into the monitor server.

    3. Create a WebSphere Business Monitor dashboard and then log in to it

      Enter your monitor model instances and add them to the dashboard. This is where you can personalize your new dashboard.

    After you have created your dashboard, use the utilities to create key performance indicators (KPIs) and subscribe to alerts. Additionally, you can add these items to your dashboard:

    • Alerts: Display alerts that notify you of defined situation events that occur at run time.
    • Diagrams: Display diagrams associated with a particular monitoring or KPI context.
    • Human tasks: Display all available human tasks. You can also perform actions on these tasks through this item. (Supported for WebSphere Process Server V6.1 only.)
    • Instances: Display the available monitoring contexts in either individual instances or user-defined groups of context instances.
    • KPIs: Displays details of KPIs, such as a KPI value relative to the defined ranges and the target, if applicable, and their status.

    These WebSphere Business Monitor utilities are supported:

    • Alerts subscription: Use to subscribe and unsubscribe to different predefined alerts. Along with subscribing to an alert, you can choose the type of notification channel associated with each alert.
    • Export values: Use to export data resulting from a specific monitor model and duration to an XML file that can be imported by WebSphere Business Modeler.
    • KPI manager: Use to define, copy, and update KPIs directly from the dashboard interface.

    You can add dimensions and reports to your dashboards by installing the optional DB2 AlphaBlox.

  5. Perform unit testing

    You now have an application that emits events for business monitoring, and a monitor model application that can consume and process the events for business monitoring. The goal of this step is to run a test of the application and evaluate its ability to generate and emit the events that WebSphere Business Monitor requires. For this testing, you will use the Monitor Test Environment, which is part of the Monitor Development Toolkit. The Monitor Test Environment runs the application and monitors the events that are produced.

    Consider the following methodology for testing the model. First, use Instances to see the monitor context application instances and the values of each instance metric. Next, use KPIs to determine whether the monitor model correctly processes the aggregate information across application instances. Then, test the visual diagram annotations. Each of these stages could involve development iterations of the application and monitor model.

    In the Servers tab, select your monitor server. Right click and select Add and Remove Projects. Select the project in the left pane and add it to the Configured Projects pane, then select Finish. Notice that the monitor server now displays "publishing" and then switches to the console tab view. Your EAR file is now deployed to the Monitor Test Environment, which means you can begin your unit test. Select the Servers tab, right-click and select WebSphere Business Monitor Dashboard. Log in and begin your testing.

  6. Deploy the monitor model (EAR file) into the monitor server after successful unit testing.

    This involves exporting the dashboard definitions, the BPEL module, and the monitor model from the Monitor Test Environment, and importing them into the production environment.

    Export the Dashboard definitions, if desired. (Remember that the ones you created in the Monitor Test Environment were really only for testing the functionality of the events and alerts.) Export the BPEL module and WebSphere Business Monitor model .ear files. Click the Open Perspective icon and select J2EE. From the Project Explorer view, right-click on your application and select Export => EAR file. On the Export dialog box, in the Destination field, enter an .ear file path and name, and then click Finish. From the Project Explorer view, right-click on your application and select Export => EAR file. On the Export dialog box, in the Destination field, enter an .ear file path and name, and then click Finish.

    Now, install the BPEL module and WebSphere Business Monitor model on a WebSphere Business Monitor production server. To accomplish this, perform the following steps:

    1. Log in to the WebSphere Business Monitor administrative console.
    2. Install and start the BPEL module.
    3. Install and start the WebSphere Business Monitor model.
    4. Create a process instance of the BPEL module.
  7. Personalize the dashboard

    You can personalize the dashboard to select what is relevant to you from the monitor server console.

  8. Monitor!

    Once deployed into the production environment, you can begin to monitor your business processes. Realize that you can also make changes to your monitoring environment from what was configured in the Monitor Test Environment. From WebSphere Business Monitor, you can personalize the dashboards created in the Monitor Test Environment, or create new ones. For example, you can alter any of the alerts to send you an SMS message instead of displaying on the dashboard.

    You can select one or more KPIs and modify the display mode (Table view, Gauge view, and so on) and the visual characteristics (color range spectrums, sizes, layout format, and so on) using the KPI Manager utility. The KPI viewer enables you to modify KPI thresholds so that you can change your success targets and evaluate various what-if scenarios.

    WebSphere Business Monitor also provides a set of configurable views for displaying human task instances to better assess workloads, identify bottlenecks, and redirect the workload to prevent backlogs. Human tasks can be BPEL tasks orchestrated by WebSphere Process Server, or standalone human tasks in the monitored system. You can view all WebSphere Process Server human-task events for different models and their properties using the Human Task Editor. You can also choose which properties to show or hide, as well as filter and define any sortable properties. The business leader can also assign, claim, release, or transfer tasks, depending on the accessibility that they have been granted.

    Figure 6. Graphical representation of the development and run time operation
    Figure 6. Graphical representation of the development and run time operation

Further reading

Further information and explicit examples with illustrations, as well as detailed step-by-step instructions can be found in the Resources section, specifically in the Business Process Management Modeling through Monitoring Redbook and the WebSphere Business Process Management Information Center. In addition, see the Clips and Tacks example for a dashboard tutorial.


Conclusion

While this article explained the steps required to monitor any System z events available in the Common Event Infrastructure, extreme detailed instructions were omitted. This was intentional: to be brief but still offer a good understanding of what is involved in this process. This article discussed what events can be monitored, the typical System z environment, Common Base Events, the Common Event Infrastructure, and the WebSphere Business Monitor architecture. Also discussed were the WebSphere Business Monitor Toolkit and WebSphere Integration Developer, which together are used to create, configure, and test the monitor model which, once deployed, will be used to consume the events that are to be monitored. Finally, you learned about configuring your dashboard to receive only the events that are of importance to you. You should now be able to configure your System z environment to obtain the information that matters to you so you can gain insight into what is happening -- real-time -- in your business processes.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Business process management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Business process management, WebSphere
ArticleID=323943
ArticleTitle=How to monitor your System z business processes
publish-date=07302008