Dimensional analysis in business activity monitoring (BAM), Part 1: Leverage WebSphere Business Monitor V6.2 to perform dimensional analysis

In this series, you'll learn about the dimensional analysis capabilities of WebSphere® Business Monitor V6.2, including how to model your data to fully exploit the powerful dimensional analysis capabilities of Monitor dashboards.

Share:

David Enyeart (enyeart@us.ibm.com), Senior Software Engineer, IBM

David Enyeart photoDavid Enyeart is the data services lead for the WebSphere Business Monitor product. He has over ten years experience building and implementing business process management (BPM) and business activity monitoring (BAM) solutions



Art Majtenyi (amajteny@clearwire.net), Software Engineer, WebSphere Consultant

 Art Majtenyi photoArt Majtenyi is a software engineer with experience in WebSphere Business Monitor, DB2, and Oracle solutions. He has 10 years experience in database access and administration, and warehouse development.



Vani Bhartipudi, Software Engineer, WebSphere Consultant

Vani Bhartipudi photoVani Bhartipudi is a software engineer with over seven years of experience in software development and support, including IBM WebSphere Business Management products.



25 August 2009

Also available in Chinese

Introduction

This article describes the WebSphere Business Monitor (hereafter called Monitor) dimensional data model, how you can model data suitable for end-user dimensional analysis, and how to use the Monitor dashboards to perform powerful analysis.


Concepts

Consider a mortgage lending firm that wants to analyze its data and identify opportunities and areas of success to fine-tune its business processes. This firm begins by understanding the building-blocks of multidimensional modeling.

Metric: The piece of data in a monitored instance that you want to analyze is called a metric in Monitor. Say you want to view a summary or average of loan amounts, so loan amount is your metric. In dimensional modeling terminology, this is also called a fact.

Measure: A measure is a metric qualified by aggregation functions across monitored instances such as average, minimum, maximum, sum, count, and standard deviation. Say you want to view the average loan amount in the past year, or you may want to see the total amount of loan money approved. Here, your measures are average loan amount and sum total loan amount, respectively. These are the values that you'll be performing dimensional analysis against.

Dimension: A dimension is a category by which you want to analyze your data. Suppose you want to view average loan amounts by various loan officers. You can define a loan officer dimension to represent this. You may want to further view average loan amounts by current year and in the approval state. You would define two more categories, or dimensions: start date and process state. You can then slice and dice your measures by any combination of all three dimensions: loan officer and start date and process state.

Dimension level: Dimensions can themselves be hierarchical, i.e. they can consist of several different levels. For example the start date dimension may have levels such as year, month, and day. These levels are powerful, enabling a user to drill-down to the level of detail needed. For example you could view loans for the current year, or each month within the year, or each day within the months. Date dimensions in Monitor reference a date or datetime metric and always have a year, month, day level hierarchy. Other types of dimensions are created by referencing a metric per level, for example a location dimension may have a country level that references a country metric, and a city level that references a city metric.

Cube: The logical grouping of measures, dimensions, and dimension levels is called a cube. Each Monitor monitoring context represents part of a business scenario that is being monitored. Each of these monitor contexts can have a cube associated with it. In fact, by default a cube will be created for each monitoring context, and by default a cube contains implicit measures for InstancesCount and implicit dimensions for CreationTime and TerminationTime that do not need to be modeled in the monitor model. You can extend the cube in the Monitor Dimensional Model for new measures and dimensions, or you can optionally delete the cube if there is no need to perform dimensional analysis on a particlular monitoring context. Cubes consume system memory, so if the cube is not required for a monitoring context, it is recommended that it be deleted.

Figure 1 represents a cube that averages the loan amount metric. Each cell in the cube contains an average loan amount value that is aggregated according to the dimensions that intersect at the cell location. With three dimensions, you can see why it is called a cube. However, there's no need to limit your analysis to three dimensions--any number of dimensions can be modeled.

Figure 1. Average loan amount cube
Average loan amount Cube

Monitor cubes are managed by an IBM Alphablox cubing engine. Alphablox ROLAP (Relational Online Analytical Processing) capabilities are used to provide cubing services on the Monitor data that is collected by monitor models. Alphablox services the dimensional queries, provides the interactive charting and drill-down capabilities, and caches dimensional data so that analysis from the Business Monitor dashboards remains responsive.


Model your data

The first step of dimensional analysis is to model your data to suit the user's analysis needs. You'll need to identify how you want to analyze the data and choose the measures and dimensions of interest. Then, you can define the measures and dimensions in the monitoring model. This is better described using an example, such as the mortgage lending example.

The rest of this article uses the mortgage lending scenario described in detail in a the developerWorks article Put new capabilities of business activity monitoring (BAM) to work, Part 3: Improved Unit Test Environment in IBM WebSphere Business Monitor Development Toolkit V6.1. The mortgage lending model is also the basis for the Monitor V6.2 dashboard showcase model, which can be installed from the Monitor First Steps page. The monitor model can be found in the Monitor installation directory and imported into the Monitor Model Editor for inspection by doing the following:

  1. Open the Monitoring Model Editor.
  2. Import the MortgageLendingBAM.zip file as a Project Interchange:
    • Select File => Import => Other => Project Interchange.
    • Click Next.
    • Browse to <WAS_INSTALL_DIR>/installableApps.wbm/showcase/model.
    • Select BetterLendingBAM_PI.zip.
    • In the projects area, select BetterLenderBAM.
    • Click Finish.
  3. Open the Business Monitoring perspective.
  4. Select Window =>Open Perspective => Business Monitoring Perspective.
  5. Open the BetterLender model in the Project Explorer by expanding BetterLenderBAM => Monitor Models, then selecting BetterLender.mm.

The Mortgage Lending process consists of several loan operations, including setting up, processing and validating the loan application, underwriting, closing, funding, post-closing and shipping. First, there is the loan set-up process, which involves creating a complete loan application. This process is frequently automated, and in the sample model, it has been created as a subprocess called Automated Loan Setup Process. Once the loan application is complete, the next step is to validate and process the application, along with underwriting it. Other steps include closing the loan, funding the loan, post-closing work and finally shipping the loan. The sample monitor model uses metrics to analyze these steps. For example, metrics are created for the application loan amount, duration days from application to funding, completed loan amount, loan number, loan officer, date started, closing duration days, and so on.

Now, consider what types of questions users might ask:

  • A supervisor may ask, "Which loan officer processes the most applications?" Or more specifically, "Which loan officer processed the most applications this year? this month? today?"
  • A process analyst may want to know, "Which area is my bottleneck, or where are most loan applications stuck at this point in time?"
  • A financial analyst may want to know, "What is the total sum of all loans that are currently in the closing step?"

In the first instance, the metric loan officer has been defined as a dimension, in order to categorize the data by loan officer, and then view the loan amounts this officer has processed. Defining another dimension, the loan start date, will help analyze yearly data and enable the drill-down to monthly and daily data. In the second and third cases, organizing the data by a loan processing state dimension enables analysis of loan applications and loan amounts in various states of processing.

In the monitoring model, you can see a cube for the Better Lending monitoring context, MortgageLending MC Cube. You can also see the three dimensions:

  • loan officer, based on the loan officer metric
  • date loan processing started, based on the loan processing start date metric
  • lending process state, based on the loan status metric

Note that when creating a dimension level based on a date or date/time metric such as loan processing start date, upon model installation Monitor will expand this level into three levels, one each for year, month, and day. The days, months, and years for the years 2000 through 2014 are pre-loaded into the Monitor V6.2 database. You may need to add years to cover date metrics before the year 2000 or after 2014. You may also want to remove unneeded years in order to decrease the number of date dimension members that are cached in sever memory, thereby improving system performance. For more information, refer to Expanding the date dimension table for Alphablox dashboards in the Monitor Information Center.

In the monitoring model you can see many measures defined, covering the number of loans (count), and average, sum, minimum, and maximum of loan amount and process duration metrics. Figure 2 shows these dimensions defined in the Monitoring Model Editor. See a larger version of Figure 2.

Figure 2. Monitoring Model Editor view
Monitoring Model Editor view

Generate the J2EE application

Once you've defined the dimensions in the model, you need to generate the Java™2 Extended Edition (J2EE) application. To do this, complete the following steps:

  1. Right-click on the mm file and select Generate Monitor J2EE Projects, as shown in Figure 3. See a larger version of Figure 3.
    Figure 3. Generate J2EE projects
    Generate J2EE projects
    This creates three new projects in the J2EE perspective: MortgageLendingBAMApplication, MortgageLendingBAMModelLogic, MortgageLendingBAMModerator.
  2. Next you need to export this J2EE application into an enterprise archive file (EAR), which can be installed on the Monitor Server. To generate the EAR file, right-click MortgageLendingBAMApplication, and select Export => EAR file , as shown in Figure 4. See a larger version of Figure 4.
    Figure 4. Export J2EE application
    Export J2EE application
  3. Save the exported EAR file. This file is also available in the same directory as the Project Interchange file: <WAS_INSTALL_DIR>/installableApps.wbm/showcase/model/BetterLenderApplication.ear.

Next, you'll install the Monitoring Model you just created.


Install the monitoring model

To install the monitoring model, do the following:

  1. In the WebSphere Application Server administrative console navigation tree, expand Applications => Monitor Models.
  2. Click Install in the work area, and browse to your model.
  3. Make sure you create the Alphablox cubes. There are two ways you can do this. The first is to ensure that Create the Alphablox cubes (the default) is selected during the model install. (Note that the Business Monitor test environment uses this selection implicitly.) To verify this selection, do the following:
    • Select Show me all installation options and parameters.
    • Click through the pages to Select Monitor Model Alphablox options and make sure Create the Alphablox cubes is selected, as shown in Figure 5. If Alphablox is installed on a remote server, supply the server hostname, RMI port, and, if security is enabled on the remote server, the User ID and password.
      Figure 5. Create Alphablox cubes during model install
      Sample figure containing an image
    • On the Summary page, click Finish.
    • Once the install completes, click Save changes directly to the master configuration.

    The second way to create the Alphablox cubes is to add the cubes after model deployment. To do this:

    • In the administrative console navigation tree, expand Applications => Monitor Models.
    • Select the model version you installed, and under Version Properties, click Manage Alphablox Cubes, as shown in Figure 6. If Alphablox is installed on a remote server, supply the server hostname, RMI port, and, if security is enabled on the remote server, the User ID and password.
    • Click Create to create the Alphablox cubes.
      Figure 6. Create Alphablox cubes after model install
      Sample figure containing an image

Note that you can use Export Cubes to export a DB2 Cube Views XML representation of the Monitor cubes. This DB2 Cube Views XML representation of the Monitor cubes can be imported into third party products that support the Cube Views interface for OLAP analysis.

Once the Alphablox cubes are created, Monitor has created all the model artifacts - the tables, views and cubes - required for dimensional analysis. It is now time to customize your dashboards to analyze Business Monitor data dimensionally.


Analyze data in dashboards

In the Mortgage Lending scenario, two monitoring contexts have been defined. Monitor and Alphablox provide the user with analytic reporting and trending capabilities over these two monitoring contexts. Alphablox provides drill-down and slicing and dicing capabilities that can answer the types of business questions that we asked earlier. Let's do some analysis to understand who are the most productive loan officers and what are the busiest months.

To start the dimensional analysis, you need to add a dimensional widget to the Monitor dashboard. Using Business Space, you can create a Business Monitoring template that provides a common dashboard set-up for business activity monitoring (BAM). The Business Monitoring template contains an Analysis tab, which contains the two widgets supported by Alphablox analytics: Dimensions and Reports.

  1. Use the configure option of the Dimensions widget to select a starting point for analysis. Select the following:
    • Monitoring Model: Better Lending BAM Showcase (All versions)
    • Monitoring context: Better Lender
    • Available dimensions: Move Loan Officer to the Column, Date Loan Processing Started to the Row, and Measures to the Page

    This configuration enables you to analyze various measures by Loan Officer and Date Loan Processing Started. Figure 7 shows the Dimensions configuration dialog.

    Figure 7. Dashboard dimensional view configuration
    Dashboard dimensional view configuration
  2. Click OK to go back to view mode with the axis configured.
  3. Choose the measure to analyze by selecting Total Number of Application Loans from the drop-down menu at the top of the chart. By default the selected measure is displayed across all dimensions; this is the top level of the cube.
  4. To see how many loans were originated by year, double-click the bar in the chart and select the Date Loan Processing Started dimension. The results are shown in Figure 8.
    Figure 8. Total number of application loans by year
    Number of loans by year
  5. To see how many loans were originated by officer and by year, double-click a bar in the chart again, and this time select the Loan Officer dimension. The results are shown in Figure 9.
    Figure 9. Total number of application loans by year and loan officer
    Number of loans by year and loan officer

You can continue drilling up and down to better understand your business data. You can arrange dimensions and measures on the dashboard however you like to understand various aspects of your business. Additionally, you can use different chart types to tailor the presentation of the data to suit your needs.

While the values that you see in the dimensional widget represent aggregate values, there are times when you'd like to drill-down even further to see the details behind a measure. For example you may notice that in one month a Loan Officer has an extremely high value for Total Amount of Application Loans. In order to understand whether this high value was caused by a few extremely large mortgages, or many smaller ones, you may want to see the individual mortgages that make up this number. Monitor V6.2 added the capability of showing these underlying instances.

To enable the Show Instances feature, do the following:

  1. Put an Instances widget on the same page as the Dimensions widget.
  2. In configure mode of each widget, select the Cooperative tab and check Enable this widget to interact with other widgets.
  3. Now when you see a measure value of interest, you can right-click the value in the Dimensional grid, and choose Show Instances. The list of instances that are defined by the chosen dimensions chosen will display in the Instances widget, as shown in Figure 10.
    Figure 10. Loans for Gerald Mander in 2008
    Sample figure containing an image

Cube maintenance

With a small amount of instance data to analyze, your drill-up and drill-down requests will be very responsive. In a large production deployment, however, there may be millions of monitored instances. In these cases, your dimensional queries may start to slow down. To combat this performance degradation, there are two things you can do. First, you can cache the measure values between sessions and across users. To turn on cube caching select Monitor Scheduled Services in the administrative console and select a model. You'll see that the Alphablox Cube Caching service is Suspended by default after model deployment. You can select to Resume the Alphablox Cube Caching service and specify how often and when to refresh the cube cache, as shown in Figure 11.

Figure 11. Monitor Scheduled Services - Cube refresh settings
Cube refresh settings

If, after turning on the Alphablox Cube Caching service, performance is still a problem and your Monitor database is DB2, you can enable the Cube Summary Table Refresh service. The Cube Summary Table Refresh service causes the database to calculate and persist measure values across dimensions in the background, providing the user with responsive dimensional analysis even when there are millions of instances being monitored. In Part 2 of this series, we'll focus on how to set up the cube summary tables.


Summary

In this article, you learned how you can leverage Monitor V6.2 to perform dimensional analysis. You learned how to model data, install the Monitor model, and customize dashboards to gather business insight from your Monitor data. In Part 2, you'll learn how to set up the cube summary tables.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=423037
ArticleTitle=Dimensional analysis in business activity monitoring (BAM), Part 1: Leverage WebSphere Business Monitor V6.2 to perform dimensional analysis
publish-date=08252009