![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
In this article, you'll learn how you can integrate high-level and low-level WebSphere Business Monitor (hereafter called Monitor) models and monitor long-running processes that can take many years to complete. You'll follow a scenario as a developer uses the Monitor model editor to integrate two types of monitor models: a high-level model, created using the Monitor toolkit, that implements monitoring of a long-running state machine with low-level models generated from WebSphere Integration Developer. The solution handles a business process that can last multiple years, in some instances never completing. The solution has a overarching state machine whose states are stored in the database with transitions between states implemented by BPEL-based business processes. Some of these business processes could be long-running BPEL-based processes, while others may be short-running BPEL-based processes.
The model editor is a component of the WebSphere Business Monitor development toolkit, which is packaged with WebSphere Business Monitor, and installed in WebSphere Integration Developer. The Integration Test Client is used to test the application in WebSphere Integration Developer, and Business Space dashboards are used to display the results.
The Texas Education Agency (TEA) provides certification to educators in the state of Texas from elementary schools to high schools and vocational and trade schools. Certification requires TEA to ensure educators are trained at educator preparation programs and that educators take the required tests from approved testing services for their subject areas. TEA must also conduct criminal background checks, collect payments for fingerprinting and certifications, and issue certificates to teach in the state of Texas.
This article uses a real-life scenario for Educator Certification that the Texas Education Agency (TEA) is implementing using IBM business process management solutions. TEA has selected WebSphere Business Monitor for its service-oriented architecture (SOA) business monitoring capabilities. TEA plans to use Monitor to monitor long-running business processes in the Educator Certification (EdCert) application and to provide key performance indicators (KPIs) in visual Web 2.0 dashboards for TEA business analysts and users
EdCert business monitoring requirements
TEA will use WebSphere Business Monitor to monitor EdCert high-level long-running processes that manage how educators obtain certifications to teach public school in Texas. These processes can take many years to complete. For example, a student can begin the process while they are in college, and must complete courses, take tests, and undertake student teaching assignments as part of their requirements for certification. A student could work partway through their requirements, take time off, then come back to complete the process, or decide to work in another field after completing their requirements (but before being certified) then come back years later to apply for certification.
The monitoring makes sense for the processing discrete business processes (for example, completion of individual test requirements or applying for a certification) but also for the overarching certification process, which can last years. The overarching process for completion of a certification is implemented as a state machine model. Discrete business processes are implemented in BPEL. When a business event happens for a given educator in the overarching process, a BPEL-based business process is started to handle it. That BPEL process may last for a finite time from a few seconds to as much as eight weeks, and then transitions the overarching process to another stable state in the state machine when it completes.
Figure 1 illustrates the state machine concept that was used in the implementation of EdCert high-level long-running processes.
Figure 1. EdCert state machine concept
In general state machine states are saved in a database and transitions between states are implemented as BPEL processes. Some BPEL processes can be long running and can update the state machine state not only at the end of the process, but in the middle of the process (for example, the educator ID verification and account set-up process). You can monitor BPEL processes as designed, but this article explains monitoring across many processes.
Based on this state machine high-level design pattern, TEA provided the following requirements for WebSphere Business Monitor:
- Correlate and monitor high-level business processes based on Educator Id and, in some cases, on Educator Objective Id. These identifiers are used to uniquely identify educators and their certification objectives. (An educator may pursue more than one certification).
- Capture the time of all significant STM state changes. It is useful for TEA to know, for example, the time it takes an educator to complete each required step for certification.
- Capture some additional business measures from BPEL processes. For example, the number of Educator Id verification attempts.
- Provide KPIs based on:
- Time duration between important states (milestones) in the state machine
- Additional business measures.
- Provide a dimensional model (described in [2}) for business analysis with the capability to drill down to the business instance (Educator) level.
- Configure Business Space dashboards that visualize KPIs, dimensional model and business instance metrics for TEA business analysts and management.
In this article, we'll use the subset of the Certification via Program (CvP) high-level business process that contains the first three states of the state machine for the process:
- Educator verification begins: An educator is first entered into the EdCert system by the educator's university counselor. This begins the process, which requires the educator to verify their identity in the EdCert system so that they can have a user account created for them.
- Educator verification complete: The educator has completed the verification process with EdCert regarding the information entered for them by their counselor.
- The educator account is ready: Based upon the educator's information entered by their counselor and verified by the educator, an identity (username) and user account for the educator has been created and the educator can now access the EdCert system.
These states are set in two BPEL processes, EdIdVerifyLR and AccountReady, that are implemented in an EdCert global module deployed to WebSphere Process Server V7.
To implement the first requirement from the previous section, we decided to use and extend the monitor model design pattern "Events linking high-level and low-level monitor models" from [1], as shown in Figure 2.
Figure 2. Events linking high-level and low-level monitor models
[1] describes a scenario in which a developer uses the monitor model editor to integrate two monitor models: a high-level model created in WebSphere Business Modeler and a low-level model generated from the WebSphere Integration Developer BPEL application that implements the business process described in WebSphere Business Modeler. In our case, the high-level model is not created in WebSphere Business Modeler, but completely implemented in the monitor model editor. As described in [1], the two monitor models communicate through custom events that are sent by the low-level model and received by the high-level model.
Low-level models generated from the BPEL application are correlated by process instance Ids that are different for several BPEL applications that contribute to a long-running state machine instance. We need to have a high-level model that represents one instance of the long-running state machine that is correlated with low-level models using Educator Id as the instance key, as shown in Figure 3.
Figure 3. EdCert state machine monitoring concept
Developing the low-level monitor model
This section describes the steps required to generate low-level monitor models from a BPEL application.
Step 1. Set the events to be emitted from the BPEL process or its activities
In the WebSphere Integration Developer BPEL editor, we enabled Common Event
Infrastructure (CEI) events (as described in [5])
in the EdVerifyIdLR BPEL for the BeginVerifying Invoke activity
WhileNotVerified loop to monitor completion of the loop ("Verification
Completed" state), and in the AccountReady BPEL for the Complete Id
Verification Invoke activity ("Account Ready" state), as shown in Figure
4. We also enabled events on value changes in the BPEL variables
attemptCount and
isVerified.
Figure 4. Set the events to be emitted from the BPEL application
Step 2. Generate a low-level monitor model from a BPEL application module
- In WebSphere Integration Developer, right-click on the GlobalModule and select Generate Monitor Model, then specify a new monitor model project name and model name.
- On the next screen, shown in Figure 5, select Include only the
events that have been turned on in the application, then
expand the two BPEL applications and select the same activities, BPEL
variables and events.
Figure 5. Generate the monitor model from the BPEL application
- On the following screen, change Implementation to Monitoring
context for the GlobalModule and to Event group
for other events, as show on Figure 6. The generated monitor model is
open in WebSphere Integration Developer Monitor perspective.
Figure 6. Choose what to monitor
Step 3. Add instance metrics to the generated monitor model
Add instance metrics to the generated monitor model to capture the state machine state change time from BPEL events and values from event payloads, such as EducatorId and "Verify Attempt Count," as shown in Figure 7.
Figure 7. Add instance metrics to the generated monitor model
Step 4. Create the XML event definition for the models to share
Refer to [1] for information on how to define a new event type in the Global_Monitor project, and define a new VerificationEventType, as shown in Figure 8, and a VerificationEvent.xsd for passing information from the low-level model to the high-level model.
Figure 8. Define VerificationEventType
Step 5. Create the outbound event in the low-level model
Refer to [1] for information on how to define new outbound events in the Global_MM model for passing information from the low-level model to the high-level model, as shown in Figure 9.
Figure 9. Define outbound events
Developing the high-level monitor model
In this section, we'll describe the steps for developing the high-level monitor model.
Step 1. Create a new monitor model
Refer to [1] for information on how to create a new monitor model CvP_HLMM.mm under the Global_Monitor project, as shown in Figure 10.
Figure 10. Create the high-level monitor model
Step 2. Create the inbound event in the high-level monitor model
Refer to [1] for information on how to create an inbound event in the CvP_HLMM.mm monitor model, as shown in Figure 11.
Figure 11. Create inbound events in the high-level monitor model
Step 3. Define an instance key for the high-level monitor model
- Define the model Key field as EducatorId and populate its value from
the Verification_Begin_Inbound_Event, as shown on Figure 12.
Figure 12. Define instance key for the high-level monitor model
- Define a correlation expression called
Verification_Begin_Inbound_Event and set the event to create a new
monitoring context if there are no instances with the same EducatorId,
as shown in Figure 13.
Figure 13. Setting the event to create a new monitoring context
Step 4. Define instance metrics in the high-level model
- Define model instance metrics and populate their values from inbound
events (see Listing 1 for example).
Listing 1. Instance metricsBegin_Verifying_Time = Verification_Begin_Inbound_Event/Verification_Begin_Info/verify:EventTime Identity Verified End Time = Verification_End_Inbound_Event/Verification_End_Info/verify:EventTime
- Define model instance metrics that represent time intervals between
state machine states. For example, for verification duration metrics
use the expression shown in Figure 14.
Figure 14. Define verification duration metrics
Step 5. Define KPIs in the high-level model
Define KPIs in the high-level model, as shown in Figure 13 (refer to the Defining key performance indicators (KPIs) section of [2] for more information.
Figure 15. Define KPIs
Step 6. Define a dimensional model in the high-level monitor model
Define a dimensional model in the high-level model, as shown on Figure 16 (refer to the Defining Dimensional Models section of 2 for more information).
Figure 16. Define a dimensional model
To deploy the new monitor models, complete the following steps:
- To ensure that the low-level monitor model Global_MM is still in sync with the BPEL applicatio, right-click one the Global_MM.mm in the Project Explorer view and select Synchronize with Application.
- Switch to the Business Integration perspective and select Project => Clean, then select the BPEL application project to rebuild.
- Right-click on one of the monitor models in the Project Explorer view
and select Generate Monitor J2EE Projects. Do the same with the
other monitor model, as shown in Figure 17.
Figure 17. Generate monitor J2EE projects
- After you've generated projects for both monitor models, go to the Server view and right-click WebSphere Business Monitor server 7.0 on WebSphere Process Server and select Start.
- After the server is started, right-click the server and select Add and Remove Projects. The two enterprise application projects for the monitor models (CvP_HLMMApplication and Global_MMApplication) show up in the list of available projects. Click each of them and select Add. Do the same for BPEL application GlobalApp.
- Click Finish to add the three projects to the server.
Follow the steps in this section to test the monitor models.
Step 1. Configure Business Space dashboards
- From the Server view, right-click WebSphere Business Monitor server 7.0 on WebSphere Process Server and select Launch => Business Space.
- Log in to Business Space with a username and password of
admin. - Click Manage Spaces and then Import to import the EdCert.zip business space configuration file. Alternatively, you can create and customize the EdCert business space and its pages yourself. For more information, refer to [3].
- Click the EdCert business space to see the four pages defined
for this space, as shown in Figure 18:
- The EdCert KPIs page is used for KPIs dashboards that are populated based on the high-level monitor model.
- The HL Instances page is used to monitor when and how high-level model instances are created and instance metrics are populated.
- The Global Process Instances page is used to monitor low-level model instances (needed for testing only)
- The EdCert dimensional model
Figure 18. EdCert Business Space – KPIs page
- To verify the widget settings, go to the Edit page and then open the options menu in the top right corner of the widget and select Edit Settings.
- Switch to the Business Integration view and select the global project
properties, then select Business Integration => Integration Test
Client, and make sure that WebSphere Business Monitor
server 7.0 on WebSphere Process Server is selected as a
default target, as shown in Figure 19.
Figure 19. Integration Test Client settings
- Open the Global Assembly diagram, right-click on the
EdIdVerifyLR BPEL component and select Test Component in
Isolation. In the Integration Test client, shown in Figure 20,
specify some test values for ecPersonId and
educatorObjectiveId, and click Continue.
Figure 20. Start testing in the Integration Test client
- Execution of the BPEL will stop several times on the Emulate
command for activities that are emulated by the Integration Test
client. Click Continue when this happens. When execution stops
on the getVerifyIdentityLoopControl operation, enter values for
timeoutDays and maxRetries, as shown in Figure 21,
and click Continue.
Figure 21. Integration Test client – continue testing
Step 3. Verify that events are received by WebSphere Business Monitor and that dashboards updated
- In the EdCert business space, click the Global Process
Instances tab, as shown in Figure 22, to make sure that events
emitted from BPEL global app are received by the Global_MM monitor
model.
Figure 22. EdCert business space – global process instances
- On the HL Instances tab, shown in Figure 23, make sure that
events are emitted from the Global_MM monitor model and received by
the CvP_HLMM Monitor model.
Figure 23. EdCert business space – HL instances
On the EdCert KPIs tab, shown in Figure 24, you can see how the CvP_HLMM monitor model calculated the KPIs actual value based on instances metrics.
Figure 24. EdCert business space – KPIs
- On the EdCert Dimensional Model tab, shown in Figure 25, you
can see how the CvP_HLMM monitor model calculated a dimensional 3D
graph based on instances metrics.
Figure 25. EdCert business space – dimensional model
In this article, you learned how to monitor long-running processes that can take many years to complete using WebSphere Business Monitor V7. You should have an understanding of the described design pattern that connects two types of monitor models when low-level models monitor events emitted from BPEL applications and pass key events to a high-level model that correlates these events and enables monitoring of a long-running process implemented as a state machine. Using the WebSphere Business Monitor development toolkit in WebSphere Integration Developer, you should be able to implement this design pattern for your own long-running processes.
- [1] Put new
capabilities of business activity monitoring (BAM) to work, Part 6:
Integrating high-level and low-level monitor models
(developerWorks)
- [2] Developing monitor models (WebSphere Business Monitor V7
Information Center)
-
[3] Monitoring with Business Space dashboards (WebSphere Business
Monitor V7 Information Center)
-
[4]
WebSphere Business Process Management V7 Production Topologies
(IBM Redbook)
- [5] Monitoring Events (WebSphere Integration Developer V7
Information Center)
-
[6] Achieve end-to-end business visibility with IBM Business Monitor
V7.5 (developerWorks): This three-part series describes the new
global monitoring context feature added in WebSphere Business Monitor
V7.5, which makes implementation of the design pattern described in this
article even easier.
-
developerWorks BPM zone
-
IBM BPM
Journal
Dr. Alexander Solovey is a Certified IT Architect in the IBM Software Services for WebSphere, Business Partner Services Practice. He is author of several publications on the topic of model-driven application execution.
Majeed Arni works in the Business Performance and Service Optimization for WebSphere organization. He is co-author of the developerWorks article "Handle industry-standard XSDs in Java using IBM Rational Software Architect."
Michel A. Ellis is lead architect on the Educator Certification project at the Texas Education Agency, where he works on business process modeling, service-oriented architecture (SOA), and Java Enterprise Edition (JEE) technologies. Michel has been working with Java and JEE technologies for almost 15 years. He enjoys hiking, traveling and listening to music.
