Running large-scale simulations with WebSphere Operational Decision Management V8

Run millions of scenarios in a timely manner

This article focuses on the large-scale simulation support introduced in IBM® WebSphere® Operation Decision Manager V8 to allow execution of simulations containing hundreds of millions of use cases in a timely fashion. We'll explain how a large-scale simulation is supported, the process to follow to create a large-scale simulation for a rule project and then guide you through a complete example of running a simulation with 120 million scenarios stored in a relational database. This content is part of the IBM Business Process Management Journal.

Sebastian Brunot (sebabrun@uk.ibm.com), Software Developer, IBM

Sebastian Bruno photoSebastien Brunot is a Software Engineer in the Hursley development laboratory. He has 11 years of experience in the Java and testing fields. He was in charge of the design and the development of the large-scale simulation support in WebSphere ODM V8.



Jose De Freitas (dfreitas@uk.ibm.com), System Tester, Operational Decision Management, IBM

Jose De Freitas photoJose de Freitas is a member of the IBM Operational Decision Management System Verification Testing (SVT) team. In this capacity, he was involved in the testing of the WebSphere ODM large-scale simulation feature. Jose previously worked in Business Process Management (BPM), in both lab and customer-facing roles, where his responsibilities included the simulation of complex business processes.



26 September 2012

Also available in Chinese

Background

Changing business rules in complex processes involving large numbers of entities, such as where customers or transactions number in the millions, is a risky prospect. It is therefore important to analyse the impact of those changes before they are implemented in the real world. Using simulation as a tool to achieve this goal is effective, but for simulations with very large volumes of input data, which we refer to as large-scale simulations, simulation results may not be obtained in a timely manner. To address this issue, WebSphere Operational Decision Manager (WebSphere ODM) V8 introduced the capability to partition simulations into separate units capable of running in different execution threads. Each execution thread carries out part of the simulation in parallel with other parts. Results are aggregated when the last of the executing simulation parts reaches completion.

Simulations in WebSphere ODM

In WebSphere ODM, a simulation is defined as the combination of:

  • A list of scenarios, where a scenario is in turn defined as the set of input parameter values required for a ruleset to run during a simulation. Each scenario in the list is identified by its index: the first scenario of the list has an index of 0, and the last one has an index of number of scenarios – 1.
  • A set of key performance indicators (KPIs) to be calculated using the inputs and outputs of the execution of each scenario in the list.

The Scenario Service Provider (SSP), the runtime component in charge of running a simulation, uses a sequential algorithm consisting of the following steps to process each simulation execution:

  1. Deploy the ruleset to be used in the simulation using a temporary name.
  2. Create and initialise the Scenario Provider, which is the component in charge of providing the list of scenarios.
  3. Create and initialise the components in charge of calculating the KPIs for the simulation.
  4. Iterate through the list of scenarios beginning at index 0. For each iteration, the SSP:
    • Uses the Scenario Provider to retrieve the scenario for the current index.
    • Notifies the KPI calculator that a scenario is going to be run and provides the calculator with the scenario data.
    • Executes the ruleset using the input parameter values provided with the scenario.
    • Notifies the KPI calculator that the scenario has finished running and provides the calculator with the output of the execution.
  5. Un-deploy the ruleset, retrieve the KPI final values from the KPI calculators and return an execution report that includes those values.

The time that it takes to calculate a KPI in the manner described above is directly proportional to the number of scenarios in the simulation. This is because the SSP uses a sequential algorithm to run the simulations. To run large-scale simulations (with millions of scenarios) in a timely fashion, you need to have a way of executing the scenarios and calculating the KPIs in a parallel manner.

In WebSphere ODM V7.5, the only option that you have to run scenarios in a parallel manner for large-scale simulations is to define different simulations on the same ruleset and then to manually consolidate the KPI results of each simulation. This is time-consuming and not very convenient. To address this issue, WebSphere ODM V8 introduces a new capability for performing large-scale simulations. The SSP now takes care of executing the scenarios in parallel and consolidating KPI results using a new set of components that are defined in the public API of the product.


Large-scale simulation support in WebSphere ODM V8

Starting with WebSphere ODM V8, the Scenario Service Provider supports a new parallel execution mode that reduces the time it takes to run large-scale simulations and to calculate the corresponding KPI values.

While the sequential execution of a simulation previously operated on a single list of indexed scenarios, the new support for large-scale simulations operates simultaneously on multiple lists of scenarios. For example, with WebSphere ODM V7.5, to run a simulation of 10 000 000 scenarios, the SSP had to iterate through a list of scenarios, starting from 0 right through to 9 999 999. With the new capabilities of WebSphere ODM V8, it is now possible for the SSP to run the same simulation iterating in parallel through, for example, five lists of scenarios with indexes ranging from 0 to 1 999 999, 2 000 000 to 3 999 999, 4 000 000 to 5 999 999, 6 000 000 to 7 999 999 and 8 000 000 to 9 999 999. Each of theses sublists of scenarios is referred to as a simulation part in the SSP documentation and public API.

The list of simulation parts that run concurrently in separate threads is provided at runtime by a new extension of the Scenario Provider component called the Parallel Scenario Provider.

Because the SSP concurrently runs each simulation part in a dedicated thread, it requires a new type of component to consolidate the partial KPI results obtained for each simulation part, and to include the consolidated values in the final report. This new component is called the KPI Result Aggregator.

When running in parallel execution mode with these two new components, the Scenario Service Provider performs the following new execution algorithm:

  1. Deploy the ruleset to be used in the simulation using a temporary name.
  2. Create and initialise the Parallel Scenario Provider, which is the component that provides the simulation parts. Unlike the scenario provider from WebSphere ODM V7.5, the parallel scenario provider supports multiple instantiation and parallel execution of its multiple instances.
  3. Retrieve a list of simulation parts from the Parallel Scenario Provider. Instead of running the list of scenarios in sequence, the SSP creates threads to concurrently run the simulation parts.
  4. Create and initialise the KPI result aggregator. This component not only performs the aggregation of the partial KPI results calculated for each simulation part, but also provides the information needed by the SSP to create and initialise the KPI calculators to use for each simulation part.
  5. Create worker threads for the sequential execution of simulation parts.
  6. Retrieve the partial KPI result from each simulation part as it reaches completion and forward its value to the KPI result aggregator.
  7. After completion of all simulation parts, retrieve the consolidated KPI value from the KPI result aggregator and return a report that contains this value.

Managing resources

To manage the amount of CPU resource that is assigned to the execution of parallel simulations, the SSP now supports resource allocation policies. These policies are configured in the deployment descriptor of the SSP web application (or using the SSP public API when the simulation is run using a local J2SE SSP service).

A default resource allocation policy, which behaves as described below, is provided with the product:

  • The default resource allocation policy assigns one execution thread to each simulation part using a fixed-size thread pool and a queue where simulation parts wait when there isn't any available thread in the pool. The pool size can be configured as part of the default resources allocation policy.
  • The default resource allocation policy can be customised to assign to each parallel simulation a maximum number of threads that is less than the pool size. This prevents one parallel simulation from using all the resources from the pool while other simulations wait.

Custom resource allocation policies are supported and can be developed by implementing an interface defined in the SSP public API. For each simulation execution request entering the SSP, the custom policy needs to define:

  • The maximum number of threads to allocate for the execution of the simulation.
  • The priority of the simulation. Three priority levels are supported: HIGH, NORMAL and LOW. The priority level of a simulation defines the priority of the threads created to run its parts.

Limitations

WebSphere ODM V8 does not support the running of simulation parts over multiple servers. This limitation did not impact our simulations involving in excess of 100 million scenarios. In our case, scenario retrieval (from the database) was clearly the bottleneck.

Another limitation, which may impact Linux™ users, is a J9 JVM limitation related to the translation of Java™ thread priorities to Linux OS process priorities ("nice" levels). This limitation, which causes all WebSphere work manager thread priorities to be mapped to the same "nice" level (0), means that in Linux environments all simulation jobs execute with the same priority.


Architecture of the parallel simulation feature

As with WebSphere ODM V7.5, the component in charge of providing scenarios for the simulation and the components in charge of calculating the KPIs are defined in a custom scenario suite format, which is authored using the testing and simulation tooling in Rule Designer. To define a scenario suite format that supports parallel execution of simulation parts, the SSP public API introduces two new interfaces in WebSphere ODM V8: IlrParallelScenarioProvider and IlrKPIResultAggregator.

IlrParallelScenarioProvider

ilog.rules.dvs.core.IlrParallelScenarioProvider is a new scenario provider interface that supports the parallel execution of simulation parts. This interface extends the existing ilog.rules.dvs.core.IlrScenarioProvider interface and defines a new method, getScenarioSuitePart(), that returns the list of parts to be used for parallel execution. This interface also defines a new contract at execution time. When running a parallel simulation, the life cycle of the parallel simulation provider in the SSP is as follows:

  1. The first instance of IlrParallelScenarioProvider is created by the SSP using reflection, through a no args constructor.
  2. The initialize(IlrScenarioSuiteExecutionContext) method is then called by the SSP on this first instance, with a scenario suite execution context whose getCurrentScenarioSuitePart() method returns null.
  3. When the initialisation has been performed, the SSP then calls the new getScenarioSuiteParts() method to retrieve the list of scenario parts for the simulation.
  4. Once the list of scenario parts has been retrieved, the SSP calls the getScenarioCount() method to retrieve the total number of scenarios in the simulation.
  5. For each part of the simulation, the SSP then performs the following operations:
    1. Creates a new instance of the parallel scenario provider using reflection.
    2. Calls the initialize(IlrScenarioSuiteExecutionContext) method of the parallel scenario provider instance using a scenario suite execution context whose getCurrentScenarioSuitePart() method returns the current part.
    3. For each of the indexes in the range defined by the part, the SSP then calls the getScenarioAt(int) method of the parallel scenario provider instance, starting from the first index and finishing with the last one.
    4. After all scenarios in the part have been run, the SSP calls the close() method on the parallel scenario provider instance.
  6. When all the parts of the simulation have been run, the SSP calls the close() method on the first parallel scenario provider instance.

IlrKPIResultAggregator

ilog.rules.dvs.core.IlrKPIResultAggregator is the new KPI result aggregator interface. It is used in combination with an IlrKPI implementation that is responsible for calculating the KPI for each part of the parallel simulation. The IlrKPI implementation is referenced by the KPI aggregator and is obtained via the getKPIClassName() method of IlrKPIResultAggregator. At runtime, the SSP creates one instance of the IlrKPI implementation for each simulation part (using reflection) and uses it to perform a standard non-parallel computation of the KPI for the simulation part. When the execution of a simulation part is completed, the SSP retrieves the partial KPI calculation using the getKPIResult() of the IlrKPI instance and notifies the IlrKPIResultAggregator instance by calling its add(IlrKPIResult) method. When all the partial results have been added to the KPI result aggregator, the SSP then calls the getKPIResult() method of the aggregator and adds this final result to the execution report.

Concurrent execution

To implement concurrent execution of the simulation parts, the SSP relies on two different solutions, depending on the runtime environment:

  • When running the SSP inside WebSphere Application Server, a WebSphere Application Server Work Manager is used to create the execution threads. Work managers allow you to configure thread pool properties using the WebSphere Application Server Administration Console.
  • When running in other supported application servers, or as a pure J2SE component, the SSP uses a J2SE thread pool.

Methodology

The process of creating and configuring the artifacts required for running large-scale simulations that meet performance requirements is not trivial. For this reason it's important to have a set of guidelines oriented towards ensuring the success and repeatability of this task. The guidelines that we offer below are not intended to be prescriptive, but to be used as a starting point in this process.

Establish the business goal

Before initiating the simulation effort, a relevant business goal needs to be established. If a business goal is 1) measurable and 2) supported by business rules, it may become a candidate for business rules simulation. For example, a relevant business goal could be to reduce the cost of a certain type of financial transaction. A more specific goal associated with reducing the cost of financial transactions might be to "reduce the number of transactions that require manual intervention." Assuming that business rules decide when manual intervention is required, changes to those rules or their input parameters may generate a number of different scenarios, the impact of which may not be easily determined without the use of a simulation tool.

Determine whether the simulation approach is applicable

In the first guideline, we established that the business objective may be partially or fully achieved through the change of business rules. If the impact of those rule changes is unknown, cannot be determined in a trivial way, or is thought to be sufficiently large that it outweighs the cost of performing simulations, then it is justified to use a simulation tool to better understand the risks of such changes.

Define how business goals are measured

Having determined the business goal and the applicability of the simulation approach, we now need to establish how the business goals should be measured. For example, to determine whether or not changes to business rules have reduced the number of transactions requiring manual intervention, we only have to count the manual transactions. In some cases, a simple metric like this one would be sufficient. However, in most cases, it's more likely that we'll also need to consider other "hidden" metrics. In our example, if we considered the manual transaction count in isolation, then a scenario where we have no manual transactions at all would be ideal. The downside is that our risk exposure would be much greater, which would effectively increase the cost of the transactions. For this reason, we also need to find a way of measuring the increased exposure as a result of less manual verification. For example, we could measure the increase in the total transaction value of the transactions that are not subject to manual verification. These metrics (manual transaction count and sum of non-manual transaction amounts) constitute the KPIs of our simulation (this is a somewhat simplistic example, but it serves to illustrate our point).

Set simulation goals

Having established a business goal, the requirement for setting additional goals for the simulation is often questioned. Notwithstanding, we've found that it's generally advantageous to word the goals in a way that is closer to the task at hand. In our example, wording the main simulation goal as "to establish the business rule changes that result in a favourable manual verification/exposure ratio" conveys a greater clarity of purpose for the task to be undertaken (assuming we know what "favourable" means) .

Define the scope of the simulation

Unlike optimisation approaches, when we perform simulations we are not seeking the best solution but, rather, an improved one. We do so by comparing the results of simulations to the status quo, or to the results of other simulations. This is an unbounded process that, for practical reasons, needs to be constrained. A good way to achieve this goal is to carry out the large-scale simulation as a formally managed project.

There are many factors that impact the amount of time and effort that is put into a simulation, including:

  • The number of "what-if" scenarios that are to be considered.
  • The performance and scalability characteristics of the hardware and software environments where the simulation is performed.
  • The time and cost of the development effort if this is the first time that the simulation artifacts (such as scenario providers and KPIs) are being developed.
  • How the input data is to be obtained, formatted and accessed.
  • The use of generated data versus real data from a database.
  • The number of input scenarios.

When you define the scope of the simulation, you need to take into account these and other factors, as well as project deadlines, time and resource constraints, and the expected benefit to be derived from the simulation effort.

In our example, assuming that we are simulating 120 million scenarios based on historical data, we should expect extensive time-consuming database design and tuning. We should also anticipate that each simulation run will take a relatively long time to complete. Thus, we might decide to restrict the number of runs (number of what-if scenarios) to a figure that is manageable within the project timeframe. It's important that we set this boundary because, in practice, there is always the temptation to continue looking for improvements. If we're not satisfied with the simulation results, or if the results are inconclusive, it's probably best to justify another simulation project with a broader scope and more time and resource.

Define how KPIs are calculated

When we defined our business and simulation goals, we identified the KPIs that would enable us to measure those goals. Here you specify how those KPIs are to be calculated. In our example, this is fairly straightforward: count of transactions that require manual intervention and sum of transaction amounts for those that don't. In this case, the result of each simulation part needs to be consolidated into a single value. For example, if we had a parallel simulation consisting of two parts and the manual transaction count for part 1 was 20 and for part 2 was 30, then the total manual transaction count would be 50. However, the consolidation or aggregation of the results of each simulation part is not always as trivial as this example implies. If you were dealing with averages, for example, then the "straight" average of all the parts would not produce the correct results. Instead, you would have to use a weighted average.

It's obvious that in this step you also need to determine what data inputs and outputs are required for KPI calculation. For example, obtaining total transaction amounts necessitates that a transaction amount field be included in the scenario input data.

Determine the simulation inputs

To determine the simulation inputs, in addition to the data required for KPI calculation, you need to examine the parameters of the rulesets that will be invoked during the simulation. Other data fields may be required, for example, to determine how the data will be partitioned in a parallel simulation. Each set of inputs constitutes a scenario. Scenarios may be based on existing historical data, or they could be generated based on statistical distributions. The execution of scenarios generated at runtime is typically much faster than that of scenarios based on historical data because no IO operations are required. The choice of either approach is influenced by the business goal (and whether or not you have historical data).

If you are using historical data, you must allocate time to finding where to get the data and to extracting the necessary data fields. If you are using statistical distributions instead, you'll need to interview subject matter experts to determine the appropriate distributions and distribution parameters.

Determine how results will be interpreted and communicated to business users

It's not unusual for the results of more complex simulations, in particular simulations that use inputs that are based on statistical distributions, to be disputed (or perceived as not very credible) by stakeholders. In practice, we have found that the up-front validation, before the first simulation is run, of the business objectives, simulation objectives and KPIs with the stakeholders is effective in increasing buy-in and reducing scepticism. During this phase, it's also important to reach a common understanding of what the results mean and their range of applicability.

Feedback obtained in these sessions with stakeholders can be used to determine how results will be rendered (and the design of the KPI renderer). In addition, you may establish that additional KPIs (auxiliary KPIs) are necessary to support and validate the main results. For example, a KPI that provides the total number of all transactions could be used to establish comparisons to the KPI that gives the sum of the non-manual transaction amounts.

Determine the data partitioning approach

When using a relational database as the source of data for the scenarios, it's important to ensure that scenario data is appropriately denormalized. For performance reasons, scenario data is typically contained in a single table, with each row in the table containing all the data that is required for one scenario. In addition, when performing parallel simulations that are based on historical data, give careful consideration to how the data is assigned to each simulation part. For example, it's common to establish this partitioning based on dates (scenario or transaction date/time). Since a varying number of transactions may occur between any two given dates, simulation parts could end up with very different sizes. Unequal parts are not ideal because the simulation duration is equal to the duration of the simulation of the biggest part.

One way to ensure that date-based partitions are of equal size is through the use of SQL windowing functions (such as ROW_NUMBER() OVER()). However, our experience with this approach is that simulations end up taking much longer than other partitioning approaches. To overcome this limitation, it may be possible to assign a sequential number to each scenario and create an index on that number. Another approach might be to manually assign equally sized date intervals to each simulation part. Whatever approach you decide on, a good early validation of suitability is that you get a constant data access response time for each partition. In other words, you should be getting approximately the same response time for retrieving the first scenarios in the partition as you would when retrieving the last scenarios.

The following is a general set of recommendations for scenarios that rely on historical data:

  • Involve an experienced database administrator early in the development of the parallel scenario provider to ensure that the database is well tuned and that the SQL is optimised.
  • Use high performance hardware for the database system, with sufficient temporary space.
  • Analyse your data to determine the best partitioning strategy.
  • Avoid the use of SQL "window functions" (such as rownum() over) or any SQL that results in large temporary result sets or database sorts (if possible use an index instead).
  • De-normalise your scenario data.
  • When designing a parallel scenario provider, it's important to allow for manual input of partition ranges, especially for those cases where automatic partitioning has a significant impact on performance.
  • A steady and high CPU usage rate on the server where the SSP runs, with frequent network activity spikes, is an indication that the database is keeping up with the data requests and that the simulation run is in a stable and predictable state. Spiked CPU activity, where the gaps between the spikes increase over time, is a clear indication that the database is not coping.

Develop the scenario providers, KPIs and DVS formats

Develop the scenario providers, KPIs and formats in the manner described in Sample implementation.

Analyse simulation performance

If the simulation is running for longer than expected you may find that by varying the number of execution threads (simulation parts) or the number of rows fetched for each query, you may notice some improvements. However, a consistent pattern of low CPU usage and infrequent network activity on the server running the SSP, excessive use of database temporary space and severe degradation of database responsiveness are all symptoms that you need to improve database access logic and/or database design.


Sample implementation

The sample implementation presented in this section is based on the use case of a financial services company that needs to assess the impact that changes in regulatory policies will have on its current payment transaction processing capability. It's expected that more manual verification, which is regulated by a ruleset, will be required, but the exact extent of the increase in manual effort is not known.

The ruleset is based on a Java XOM that provides two main classes:

  • com.bank.payment.Payment, which defines a payment transaction with the following attributes:
    • amount - the amount of the payment.
    • currency - the currency of the payment.
    • debitAccount - the bank account the payment is emitted from.
    • beneficiaryAccount - the bank account the payment is made to.
    • date - the date the payment transaction was created.
    • valueDate - the value date for the payment transaction.
    • manual - a flag that signals whether the payment transaction should be manually processed.
  • com.bank.payment.AccountServices, which defines a set of services to use to retrieve information related to a bank account. Note that in the sample XOM project provided with this article, the implementation of AccountServices is a sample one that does not connect to a back office system to retrieve information.

The ruleset signature comprises an IN_OUT parameter of type Payment, whose manual flag attribute is updated by the ruleset if necessary, and an IN parameter of type AccountServices.

In order to evaluate the impact of modifications in the regulation policy, the company decides to run simulations against its database of past payment transactions. The company uses a business intelligence tool on its historical database that makes it simple to count the number and total amount of payment transactions between two given dates. The scope of the simulation is then defined as the number of payment transactions between the two dates.

After discussing with the database administrators and testing different data access strategies, the decision is made to use a single table to store the simulation data. This table, named DATA, is defined with the following columns:

Table 1. Columns of the DATA table
Column Name Column Type Comment
ID INT Primary key, generated when inserting the payment entries ordered by ascending dates
CURRENCY CHAR(3) Currency of the payment
AMOUNT DECIMAL(15,2) Amount of the payment
DEBTACC VARCHAR(27) Debit account
BENEFACC VARCHAR(27) Beneficiary account
DATE BIGINT Payment date
VALUEDATE BIGINT Payment value date

Since the primary partition key for the payment transaction data is the date, it's decided that the simulation parts should be defined as a list of dates, each simulation part containing the payments defined between two consecutive dates of that list. For example, to split a simulation on the payments made between DATE1 and DATE2 in three parts, the user uses the business intelligence tool to retrieve the two dates, DATEa and DATEb, such that there is roughly the same number of payment transactions in the database between DATE1 and DATEa (first simulation part), DATEa and DATEb (second simulation part) and DATEb and DATE2 (third simulation part).

To avoid maintaining a lengthy connection to the database, or issuing a request that tries to retrieve all the rows from the database at once, each scenario provider instance uses a cache of Payment objects that is filled when necessary, using a database request that returns the exact number of rows needed to entirely fill the cache. The size of the cache is provided as a parameter of the simulation and, if empty, the cache is refilled when a new scenario is requested by the SSP.

The SQL request used to retrieve the data (and fill the cache) is the following:

SELECT ID, CURRENCY, AMOUNT, DEBTACC, BENEFACC, DATE, VALUEDATE 
FROM DATA WHERE DATA.DATE between {0} and {1} AND ID >= {2} 
ORDER BY ID ASC FETCH FIRST {3} ROWS ONLY OPTIMIZE FOR {3} ROWS 
FOR READ ONLY WITH UR

where {0} is replaced by the date of the first payment transaction to use for the simulation part, {1} is replaced by the date of the last payment transaction to use for the simulation part, {2} is replaced by the ID of the payment that follows the last one processed from the cache and {3} is replaced by the size of the cache of Payment objects.

The following set of KPIs have been identified:

  • Total number of payments in the simulation.
  • Number of payments for which further manual processing is required.
  • Total (transaction) amount for payments for which further manual processing is required.
  • Average amount for payments for which further manual processing is required.
  • Number of payments for which no further manual processing is required.
  • Total amount for payments for which no further manual processing is required.
  • The average amount for payments for which no further manual processing is required.
  • The total execution time for the simulation (not a business KPI).

All the sample materials that you need to implement this use case care available for download with this article. The payment.zip archive contains:

  • A Java project to use to create a sample payments database (payment-database).
  • The Java XOM project (payment-xom) and the rule project (payment-processing).
  • Implementations for the parallel scenario provider and KPI components (sample-src).

To install and run the sample, complete the following steps.

Step 1: import the sample material into Rule Designer

  1. Start Rule Designer in a clean workspace, and select File => Import to open the import wizard. In the import wizard, select General => Existing Projects into Workspace, as shown in Figure 1, and click Next.
    Figure 1. Import project
    Import project
  2. In the import wizard, select Select archive file and click Browse to navigate to the payment.zip archive that you downloaded. The content of the wizard is updated with the three projects contained in the archive:
    • payment-database is a Java project that contains the sample database creation script and a Java utility used to load the database with sample data.
    • payment-processing is the rule project that defines the regulation policy (rules) for the simulation.
    • payment-xom is the Java XOM for payment-processing.
  3. Select the three projects to import, as shown in Figure 2.
    Figure 2. Select the payment projects to import
    lect the payment projects to import
  4. Click Finish to import the three projects in your workspace, as shown in Figure 3.
    Figure 3. Payment projects displayed in Rule Explorer after import
    Payment projects displayed in Rule Explorer after import

tep 2: create and populate the sample database

The payment-database project you imported into Rule Designer in step 1 contains the following artifacts:

  • create-schema.sql, the SQL script to use to create the sample database schema in your DB2 database.
  • com.bank.payment.database.DataLoader, a Java class that defines a main method that you run to populate the sample database.

If you switch to the Java perspective in Rule Designer, you can easily access the two artifacts, as shown in Figure 4.

Figure 4. Content of the payment-database project
Content of the payment-database project

Tip for creating the database

When creating the database, involve an experienced DBA. Very large databases have to be very finely tuned, and the usual configuration defaults may not be sufficient or adequate. For example, in DB2® you need to make sure that you have enough space in your storage containers, that your buffer pools and logs are adequately sized, that database configuration parameters (such as BUFFPAGE and DBHEAP) have values that are large enough, and so on.

After you've created the database schema using the create-schema.sql script, you need to add the DB2 JDBC drivers JAR to the build path of the payment-database project and then run the DataLoader class as a Java program to populate the database, providing the following six arguments to the Java program:

  • The name of the JDBC driver to use to access your database.
  • The JDBC URL to your database.
  • The user name to use to access your database.
  • The password to use to access your database.
  • The number of rows (payments) to create in your database.
  • The batch size, which is the number of payments to create in a single operation when populating the database.

Figure 5 shows an example of the parameter values we used to populate a DB2 sample database with 120 million payments. It took 2 hours and 41 minutes to load the database.

Figure 5. Run configuration to populate a DB2 database with 120M entries
Run configuration to populate a DB2 database with 120M entries

Step 3: Create the custom scenario suite format

To create a custom scenario suite format for a large-scale simulation, you need to create a new DVS project in Rule Designer, and then use the new scenario suite format wizard to define the parallel scenario provider, as well as the KPI to use with the custom format.

  1. In Rule Designer, press Ctrl+N to display the New wizard.
  2. Select Rule Designer => Decision Validation Services => DVS Project, as shown in Figure 6.
    Figure 6. New DVS project creation wizard
    New DVS project creation wizard
  3. Click Next and specify a name for your project (Payment Simulations in our example), as shown in Figure 7.
    Figure 7. Name the new DVS project
    Name the new DVS project
  4. Click Next and then Finish to create the new DVS project. The DVS Customization editor is now displayed in Rule Designer, as shown in Figure 8.
    Figure 8. DVS Customization editor
    DVS Customization editor

    (See a larger version of Figure 8.)

  5. The next step is to add a runtime configuration, which defines the target application server for Decision Center and the SSP, to the customization. To do this, click Create in the Configurations section and then select the target application server. You'll then add the payment-processing rule project to the list of rule projects (by clicking Add in the Rule Projects section and selecting the rule project, as shown in Figure 9.
    Figure 9. WebSphere Application Server V8 configuration
    WebSphere AS V8 configuration

    (See a larger version of Figure 9.)

  6. To create the new scenario suite format that will be used to run parallel simulations on the payment-processing rule project, click Create in the Formats section, which will open the New DVS Format wizard, as shown in Figure 10.
    Figure 10. New DVS Format wizard
    New DVS Format wizard
  7. Specify a name for this format (payment database in our example) and click Next.
  8. On the next screen, you define the scenario provider implementation to use for the format. To create a new parallel scenario provider, click Create, which displays the New DVS Scenario provider wizard. The important user action here, besides providing package and class names for the scenario provider components to be generated, is to check Generate a scenario provider that supports parallel execution of simulations, as shown in Figure 11. This ensures a parallel scenario provider is generated.
    Figure 11. Check the parallel execution option
    Check the parallel execution option
  9. Click Finish to generate the scenario provider and the scenario suite renderer classes. This takes you back to the New DVS Format wizard, which is now updated with the name of the new scenario provider, as shown in Figure 12.
    Figure 12. New scenario provider selected for the DVS format
    New scenario provider selected for the DVS Format
  10. Click Finish to close the New DVS Format wizard and open the DVS Format editor for the newly created format, as shown in Figure 13.
    Figure 13. DVS Format editor
    DVS Format editor

    (See a larger version of Figure 13.)

  11. To complete the definition of the new scenario suite format, you need to define the KPI to use when running a simulation with this format. To do this, click New in the KPI section of the editor to open the KPI Result Aggregator Definition dialog, as shown in Figure 14.
    Figure 14. KPI Result Aggregator Definition dialog
    KPI Result Aggregator Definition dialog
  12. The KPI Display Name identifies the KPI in the simulation reports (Payment statistics in our example). To define the implementation classes used to calculate and aggregate the KPI values, click Create next to the KPI Result Aggregator Class to display the New KPI Result Aggregator wizard, as shown in Figure 15.
    Figure 15. New DVS KPI Result Aggregator wizard
    New DVS KPI Result Aggregator wizard
  13. Two options are provided here: to create a new KPI Result Aggregator from scratch, or to create an aggregator for an IlrKPI component already defined for a non-parallel simulation type. In our example, we're going to create a new IlrKPI implementation to calculate the KPI values for each simulation part, so accept the already selected first option and click Next. This opens the New DVS KPI Result Aggregator wizard, where you'll provide the following information:
    • The IlrKPI implementation to create (com.bank.payment.PaymentKPI in our example).
    • The KPI result aggregator implementation to create (com.bank.payment.PaymentKPIResultAggregator in our example).
    • The renderer, used to display the KPI result in the simulation report in Decision Center, to create (com.bank.payment.PaymentKPIRenderer> in our example).
    • The type of KPI result that is calculated. In our example, we use the type Map, which provides a convenient way to store multiple values in the KPI result, while defining only one IlrKPI implementation and one KPI result aggregator.

    Figure 16 shows what the wizard dialog looks like when all the inputs have been provided.

    Figure 16. Completed New DVS KPI Result Aggregator wizard
    Completed New DVS KPI Result Aggregator wizard
  14. Click Finish to generate template implementation classes for the KPI components and to update the KPI Result Aggregator Definition dialog, as shown in Figure 17.
    Figure 17. KPI Result Aggregator Definition dialog
    KPI Result Aggregator Definition dialog
  15. Click OK to close this dialog and press Ctrl+S to save the payment database scenario suite format in the Scenario Suite Format editor.

Step 4: implement the parallel scenario provider components

The wizards generated templates for the scenario provider components listed below, and you'll now need to implement them:

  • com.bank.payment.PaymentParallelScenarioProvider is the parallel scenario provider.
  • com.bank.payment.PaymentScenarioSuiteResourcesRenderer is the Decision Center component that takes care of retrieving user parameters for a simulation. These parameter values are then provided to the parallel scenario provider at initialisation time.

Figure 18 displays these two Java class templates in the Package Explorer (the Java perspective in Rule Designer).

Figure 18. Details of the scenario provider implementation classes
Details of the scenario provider implementation classes

PaymentScenarioProvider implementation

The complete source code of the com.bank.payment.PaymentParallelScenarioProvider class is provided in the sample-src directory of the payment.zip archive provided for download with this article.

Let's look at the implementation details of the scenario provider. Each instance of the parallel scenario provider uses a JDBC Connection object to access the payment database. This Connection object, which connects to the database via a JDBC datasource that is stored in the JNDI directory of the SSP application server host under the name jdbc/lssDB is retrieved in the initialize(...) method of the scenario provider. The connection is closed in the close() method of the scenario provider. Listing 1 shows the database connection life cyle

Listing 1. Database connection life cycle
publicstaticfinal String DATASOURCE_JNDI_NAME = "jdbc/lssDB";
private Connection dbConnection = null;

/**
 * Callback method invoked by the runner in order to inject the execution
 * context into the component.
 */
publicvoid initialize(IlrScenarioSuiteExecutionContext context)
  throws IlrInitializationException {

  // Retrieve a connection to the payment database
  try {
    Context jndiContext = new InitialContext();
    DataSource datasource = (DataSource) jndiContext.lookup(DATASOURCE_JNDI_NAME);
    this.dbConnection = datasource.getConnection();
  } catch (Throwable t) {
    thrownew IlrInitializationException(t);
 }

  // …
}

/**
 * Close the scenario provider.
 */
publicvoid close() {
  if (this.dbConnection != null) {
    try {
      this.dbConnection.close();
    } catch (SQLException e) {
 e.printStackTrace();
 }
 }
}

The two simulation parameters entered by the simulation user in the Decision Center are retrieved from the scenario suite execution context, which is provided to the initialize(...) method by the SSP:

  • The cache size, which defines the size of the internal cache of Payment instances, is an int. Its name is CACHE_SIZE.
  • The date range, which defines the list of beginning and end dates for each simulation part, is a List<Long>. Its name is DATE_RANGES.

Both values are stored in a nested class named ScenarioProviderParameters,, which provides a factory method to retrieve the parameter values from a scenario suite context instance as shown in Listing 2.

Listing 2. Retrieving scenario parameters
publicstaticfinal String DATE_RANGES = "DATE_RANGES";
publicstaticfinal String CACHE_SIZE = "CACHE_SIZE";

/**
 * Bean that stores the scenario provider parameters
 */
publicstaticfinalclass ScenarioProviderParameters {

  privateint cacheSize = -1;
  private List<Long> dateRanges = new ArrayList<Long>();

  publicstatic ScenarioProviderParameters fromScenarioSuiteDescriptor(
    // … 
  }

  // … 
}

private ScenarioProviderParameters parameters = null;

/**
 * Callback method invoked by the runner in order to inject the execution
 * context into the component.
 */
publicvoid initialize(IlrScenarioSuiteExecutionContext context)
  throws IlrInitializationException {
  // …
  // Extract the simulation parameters from the context
  this.parameters = ScenarioProviderParameters
    .fromScenarioSuiteDescriptor(context.getScenarioSuiteDescriptor());
  // …
}

The total number of scenarios in the simulation is defined by the number of payments in the payment database between the first and the last date of the date range parameter, as shown in Listing 3.

Listing 3. Retrieving the total number of scenarios in the simulation
/**
 * Return the count of scenarios for this provider.
 * 
 * @return the count of scenarios for this provider
 * @throws IlrTestingException
 * if an error occurred while computing the scenarios count
 */
publicint getScenarioCount() throws IlrScenarioProviderException {
  // Count the number of scenarios in the database
  long firstDate = parameters.getDateRanges().get(0);
  long lastDate = parameters.getDateRanges().get(
    parameters.getDateRanges().size() - 1);
  return countRecordsBetweenDates(firstDate, lastDate);
}

/**
 * Count the number of payments between two dates in the database.
 */
privateint countRecordsBetweenDates(Long startDate, Long endDate)
  throws IlrScenarioProviderException {
  int returnedValue = 0;
  Statement stmt = null;
  ResultSet result = null;
  try {
    stmt = this.dbConnection.createStatement();
    String request = MessageFormat.format(
    "SELECT COUNT(*) FROM DATA WHERE DATE BETWEEN {0} AND {1}",
      new Object[] { String.valueOf(startDate),
      String.valueOf(endDate) });
    result = stmt.executeQuery(request);
    if (result.next()) {
      returnedValue = result.getInt(1);
    }
  } catch (SQLException e) {
    thrownew IlrScenarioProviderException(e);
  } finally {
    // … (close resultset and statement)
  }
  return returnedValue;
}

Each simulation part is defined by a start date and an end date that are consecutive in the list of dates entered by the end user in Decision Center. For a simulation part, the parallel scenario provider will return scenarios for all the payments stored in the database between the start date (included) and the end date (not included, except for the last date). Because the scenario provider is using a cache of Payment objects rather than querying the database once to retrieve all the payments for a part, it also keeps track, for each part, of the ID of the next payment to be loaded in the cache when querying the database. The method getScenarioSuiteParts() defines the parts of the simulation and stores the part parameters (start date, end date, ID of the next payment for the part) as custom data of the part using a long[], as shown in Listing 4.

Listing 4. Retrieving the simulation parts
/**
 * Returns the list of parts to be used for the parallel execution of the
 * simulation.
 * 
 * @return The list of parts to be used by the SSP to execute a simulation
 * concurrently in multiple processors or threads. For each part of
 * the list, the SSP creates a new instance of the scenario provider
 * and iterates through the range of indexes defined for the part. The
 * KPI result calculated for each part is then consolidated by an
 * instance of {@link IlrMapReduceKPI}.
 */
public List<IlrScenarioSuitePart> getScenarioSuiteParts()
  throws IlrScenarioProviderException {
  ArrayList<IlrScenarioSuitePart> returnedValue =
    new ArrayList<IlrScenarioSuitePart>();
  // A simulation part is defined as the payments between two dates
  // from the range
  List<Long> dateRanges = this.parameters.getDateRanges();
  int numberOfDates = dateRanges.size();
  int currentScenarioIndex = 0;
  for (int partNumber = 0; partNumber < numberOfDates - 1; partNumber++) {
    // A part if defined by:
    // 1) a start date
    // 2) an end date
    // 3) the ID of the next payment to retrieve from the database
    // between these two dates (in order to refill the cache of
    // payments for the part)
    long startDate = this.parameters.getDateRanges().get(partNumber);
    long endDate = this.parameters.getDateRanges().get(partNumber + 1);
    // the payments with a date equal to the end date are not
    // included in the part, except for the last part
    if (partNumber < numberOfDates - 2) {
      endDate = endDate - 1;
    }
    // count the number of scenarios (payments) for this range
    int numberOfScenarios = countRecordsBetweenDates(startDate, endDate);
    // retrieve the next record ID for this part
    int nextRecordIDForThePart = -1;
    Statement stmt = null;
    ResultSet result = null;
    try {
      stmt = this.dbConnection.createStatement();
      String request = MessageFormat.format(
        "SELECT ID FROM DATA WHERE DATA.DATE >= {0} ORDER BY ID ASC",
        new Object[] { String.valueOf(startDate) });
      result = stmt.executeQuery(request);
      if (result.next()) {
        nextRecordIDForThePart = result.getInt(1);
      } else {
        thrownew IlrScenarioProviderException("No records for date >= "
          + String.valueOf(startDate));
      }
    } catch (SQLException e) {
      thrownew IlrScenarioProviderException(e);
    } finally {
      // … (close request and statement)
    }
    // Each part contains its inclusive range as custom data, as
    // well as the value of the next record ID to use to retrieve
    // data for the part
    IlrScenarioSuitePart part = new IlrScenarioSuitePart(
      currentScenarioIndex, currentScenarioIndex + numberOfScenarios - 1,
      newlong[] { startDate, endDate, nextRecordIDForThePart });
      returnedValue.add(part);
    currentScenarioIndex += numberOfScenarios;
  }
  return returnedValue;
}

When the initialize(...) method is run for a simulation part, the scenario provider retrieves the information that defines the part (start date, end date, ID of the next payment to be retrieved from the database) in the part's custom data object (long[]). These values are stored as attributes of the scenario provider, so they can be retrieved by the getScenarioAt(...) method when it is called by the SSP, as shown in Listing 5.

Listing 5. Initializing the current simulation part
privatelong startDateForTheCurrentPart = -1;
privatelong endDateForTheCurrentPart = -1;
privateint nextRecordID = -1;
private Payment[] paymentCache = null;

/**
 * Callback method invoked by the runner in order to inject the execution
 * context into the component.
 */
publicvoid initialize(IlrScenarioSuiteExecutionContext context)
  throws IlrInitializationException {
  // …

  // If the scenario provider is initialized for a part, retrieve the
  // part parameters from the custom data of the part.
  IlrScenarioSuitePart currentPart = context.getCurrentScenarioSuitePart();
  if (currentPart != null) {
    long[] scenarioPartData = (long[]) currentPart.getCustomData();
    this.startDateForTheCurrentPart = scenarioPartData[0];
    this.endDateForTheCurrentPart = scenarioPartData[1];
    this.nextRecordID = (int) scenarioPartData[2];
    // Initialize the Payment cache
    this.paymentCache = new Payment[parameters.getCacheSize()];
  }
}

The getScenarioAt(...) method retrieves Payment instances from the cache and is responsible for (re-)filling the cache when it is empty, as shown in Listing 6.

Listing 6. Retrieving scenarios
public static final String PAYMENT_PARAMETER_NAME = "payment";
public static final String ACCOUNTSERVICES_PARAMETER_NAME = "accountServices";

private int currentCacheIndex = 0;
private int numberOfEntriesInCache = 0;
private AccountServices accountServices = new AccountServices();

/**
 * Return the scenario at specified index
 * 
 * @param scenarioIndex
 * the index of the scenario
 * @return the scenario at specified index
 * @throws IlrTestingException
 * if an exception occurred while retrieving the scenario
 */
public IlrScenario getScenarioAt(int scenarioIndex)
  throws IlrScenarioProviderException {
  // Retrieve scenario from the cache
  if (this.currentCacheIndex == 0) {
    // We need to refill the buffer
    ResultSet result = null;
    Statement getScenarioDataStatement = null;
    try {
      getScenarioDataStatement = this.dbConnection.createStatement();
      if (this.paymentCache.length > 1000) {
        getScenarioDataStatement.setFetchSize(this.paymentCache.length);
      }
      String request = MessageFormat.format(
        "SELECT ID, CURRENCY, AMOUNT, DEBTACC, BENEFACC, DATE, VALUEDATE" +
        " FROM DATA WHERE DATA.DATE between {0} and {1} AND ID >= {2} ORDER BY ID ASC" +
        " FETCH FIRST {3} ROWS ONLY OPTIMIZE FOR {4} ROWS FOR READ ONLY WITH UR",
        new Object[] { String.valueOf(this.startDateForTheCurrentPart),
        String.valueOf(this.endDateForTheCurrentPart),
        String.valueOf(nextRecordID),
        String.valueOf(this.paymentCache.length), 
        String.valueOf(this.paymentCache.length) });
      result = getScenarioDataStatement.executeQuery(request);
      this.numberOfEntriesInCache = 0;
      while (result.next() && this.numberOfEntriesInCache < this.paymentCache.length) {
        this.nextRecordID = result.getInt(1) + 1;
        Payment payment = new Payment();
        payment.setCurrency(Currency.valueOf(result.getString(2)));
        payment.setAmount(result.getBigDecimal(3));
        payment.setDebitAccount(result.getString(4));
        payment.setBeneficiaryAccount(result.getString(5));
        payment.setDate(new Date(result.getLong(6)));
        payment.setValueDate(new Date(result.getLong(7)));
        // Add the payment to the cache
        this.paymentCache[this.numberOfEntriesInCache] = payment;
        this.numberOfEntriesInCache++;
      }
    } catch (SQLException e) {
      throw new IlrScenarioProviderException(e);
    } finally {
      // … (close resultset and statement)
    }
  }
  IlrScenarioImpl returnedValue = new IlrScenarioImpl();
  returnedValue.setName(new StringBuffer("Scenario ")
    .append(scenarioIndex)
    .toString());
  Map<String, Object> scenarioInputParameters = new HashMap<String, Object>();
  scenarioInputParameters.put(PAYMENT_PARAMETER_NAME,
  this.paymentCache[this.currentCacheIndex]);
  scenarioInputParameters.put(ACCOUNTSERVICES_PARAMETER_NAME, accountServices);
  returnedValue.setInputParameters(scenarioInputParameters);

  if (++this.currentCacheIndex >= this.numberOfEntriesInCache) {
    this.currentCacheIndex = 0;
  }

  return returnedValue;
}

PaymentScenarioSuiteResourcesRenderer implementation

The complete source code of the com.bank.payment.PaymentScenarioSuiteResourcesRenderer class is provided in the sample-src directory of the payment.zip archive provided for download with this article.

The scenario suite resources renderer is a component defined by the Decision Center public API. Decision Center stores the parameters of a simulation as byte[] objects, and the scenario suite resources renderer provides the following three methods to help Decision Center process these parameters:

  • The encodeAsEditor(...) method is responsible for creating the HTML input fields that the end user uses to supply the parameters of a simulation. The values for each parameter, if they are already defined, are provided by Decision Center as byte[] objects stored in a map (parameter resources). The HTML elements created by this method are rendered by the Decision Center when a user creates or edits a simulation that uses the payment database format.
  • The encodeAsViewer(...) method is in charge of creating the HTML elements that display the simulation parameters to the end user when viewing the simulation in Decision Center.
  • The decode(...) is responsible for retrieving the simulation parameter values entered by the end user in the Decision Center and storing them in a map of byte[] objects.

tep 5: Implement the KPI components

Following are the KPI components to implement, for which templates were generated by the Rule Designer wizards:

  • com.bank.payment.PaymentKP is the component in charge of calculating the KPI values for a simulation part.
  • com.bank.payment.PaymentKPIResultAggregator is the component in charge of aggregating the KPI results of the parts and returning a global KPI result for the simulation.
  • com.bank.payment.PaymentKPIRenderer is the Decision Center component that displays the KPI result in the simulation report.

Figure 19 displays these three class templates in the Package Explorer (the Java perspective in Rule Designer).

Figure 19. Detail of the KPI implementation classes
Detail of the KPI implementation classes

PaymentKPI implementation

The complete source code of the com.bank.payment.PaymentKPI class is provided in the sample-src directory of the payment.zip archive provided for download with this article.

The KPI values are stored in attributes of the class. The values of the attributes are updated after each execution of a scenario, except for the average values, which are calculated later by the KPI result aggregator, as shown in Listing 7.

Listing 7. Calculation of KPI values
privateint totalNumberOfPayments = 0;
privateint numberOfManualProcessings = 0;
private BigDecimal totalAmountManuallyProcessed = new BigDecimal(0);
privateint numberOfAutomatedProcessings = 0;
private BigDecimal totalAmountAutomaticallyProcessed = new BigDecimal(0);

/**
 * Called by the runner after the ruleset has executed.
 * 
 * This method is called for each scenario defined in the suite. When it is
 * called, the report element is not yet available from the suite report.
 * 
 * @param scenario
 * The scenario.
 * @param request
 * The request used to execute the ruleset to test.
 * @param response
 * The response returned by executing the ruleset.
 * @throws IlrKPIException
 * if an error occurs.
 */
publicvoid onScenarioEnd(IlrScenario scenario, IlrSessionRequest request,
  IlrSessionResponse response) throws IlrKPIException {
  this.totalNumberOfPayments++;
  Payment payment = (Payment) response.getOutputParameters().get("payment");
  if (payment.isManualProcessing()) {
    this.numberOfManualProcessings++;
    this.totalAmountManuallyProcessed = 
    this.totalAmountManuallyProcessed.add(payment.getAmount());
  } else {
    this.numberOfAutomatedProcessings++;
    this.totalAmountAutomaticallyProcessed =
      this.totalAmountAutomaticallyProcessed.add(payment.getAmount());
  }
}

The KPI result is a Map that contains all the KPI values calculated during the execution, as shown in Listing 8.

Listing 8. Creation of the KPI result
publicstaticfinal String TOTAL_NB_OF_PAYMENTS =
  "Total nb of payments";
publicstaticfinal String NB_OF_MANUAL_PROCESSINGS =
  "Nb of manual processings";
publicstaticfinal String TOTAL_AMOUNT_MANUALLY_PROCESSED =
  "Total amount manually processed";
publicstaticfinal String NB_OF_AUTOMATED_PROCESSINGS =
  "Nb of automated processings";
publicstaticfinal String TOTAL_AMOUNT_AUTOMATICALLY_PROCESSED =
  "Total amount automatically processed";

/**
 * Returns the KPI result to be included in the full report.
 * 
 * This method is called after all of the parts of the simulation have been
 * executed.
 * 
 * @return The KPI result.
 * @throws IlrKPIException
 * if an error occurs while computing the result.
 */
public IlrKPIResult getKPIResult() throws IlrKPIException {
  IlrKPIResultMap returnedValue = new IlrKPIResultMap();
  returnedValue.setKPIClassName(PaymentKPI.class.getName());
  Map<String, Serializable> value = new HashMap<String, Serializable>();
  value.put(TOTAL_NB_OF_PAYMENTS, this.totalNumberOfPayments);
  value.put(NB_OF_MANUAL_PROCESSINGS, this.numberOfManualProcessings);
  value.put(TOTAL_AMOUNT_MANUALLY_PROCESSED, this.totalAmountManuallyProcessed);
  value.put(NB_OF_AUTOMATED_PROCESSINGS, this.numberOfAutomatedProcessings);
  value.put(TOTAL_AMOUNT_AUTOMATICALLY_PROCESSED,
  this.totalAmountAutomaticallyProcessed);
  returnedValue.setValue(value);
  return returnedValue;
}

PaymentKPIResultAggregator implementation

The complete source code of the com.bank.payment.PaymentKPIResultAggregator class is provided in the sample-src directory of the payment.zip archive provided for download with this article.

The total duration of the simulation execution is calculated in the KPI result aggregator, since this is the first KPI component created and the last one called to retrieve the KPI result, as shown in Listing 9.

Listing 9. Calculation of the simulation duration
privatelong startExecutionTimestamp = 0;

/**
 * Initializes this object with an execution context.
 * 
 * @param context
 * The execution context.
 * @throws IlrInitializationException
 * if an error occurs while initializing this object.
 */
publicvoid initialize(IlrScenarioSuiteExecutionContext context)
  throws IlrInitializationException {
  this.startExecutionTimestamp = System.currentTimeMillis();
}

/**
 * Returns the KPI result to be included in the full report.
 * 
 * This method is called after all of the parts of the simulation have been
 * executed.
 * 
 * @return The KPI result.
 * @throws IlrKPIException
 * if an error occurs while computing the result.
 */
public IlrKPIResult getKPIResult() throws IlrKPIException {
  IlrKPIResultMap returnedValue = new IlrKPIResultMap();
  // … 
  Map<String, Serializable> value = new HashMap<String, Serializable>();
  long durationInSeconds = (System.currentTimeMillis() -
  this.startExecutionTimestamp) / 1000;
  value.put("Total duration of the execution",
  String.valueOf(durationInSeconds) + " sec.");
  // … 
  returnedValue.setValue(value);
  return returnedValue;
}

The KPI values calculated by the PaymentKPI instances are aggregated each time a result is available, as shown in Listing 10.

Listing 10. Aggregation of the KPI results
privateint totalNumberOfPayments = 0;
privateint numberOfManualProcessings = 0;
private BigDecimal totalAmountManuallyProcessed = new BigDecimal(0);
privateint numberOfAutomatedProcessings = 0;
private BigDecimal totalAmountAutomaticallyProcessed = new BigDecimal(0);

/**
 * Adds the result of a KPI calculated for a part of the simulation, or
 * scenario suite.
 * 
 * This method is called at the end of the execution of each part.
 * 
 * @param kpiResult
 * The result of the simulation part, which is calculated by the
 * IlrKPI instance returned by the {@link #getKPIClassName()}
 * method.
 */
publicvoid add(IlrKPIResult kpiResult) throws IlrKPIException {
  Map<String, Serializable> values = ((IlrKPIResultMap) kpiResult).getValue();
  this.totalNumberOfPayments +=
    (Integer) values.get(PaymentKPI.TOTAL_NB_OF_PAYMENTS);
  this.numberOfManualProcessings +=
    (Integer) values.get(PaymentKPI.NB_OF_MANUAL_PROCESSINGS);
  this.totalAmountManuallyProcessed =
    this.totalAmountManuallyProcessed.add((BigDecimal)
    values.get(PaymentKPI.TOTAL_AMOUNT_MANUALLY_PROCESSED));
  this.numberOfAutomatedProcessings +=
    (Integer) values.get(PaymentKPI.NB_OF_AUTOMATED_PROCESSINGS);
  this.totalAmountAutomaticallyProcessed =
    this.totalAmountAutomaticallyProcessed.add((BigDecimal)
    values.get(PaymentKPI.TOTAL_AMOUNT_AUTOMATICALLY_PROCESSED));
}

The average values are calculated with the final KPI result, as shown in Listing 11.

Listing 11. Calculation of the final KPI result
/**
 * Returns the KPI result to be included in the full report.
 * 
 * This method is called after all of the parts of the simulation have been
 * executed.
 * 
 * @return The KPI result.
 * @throws IlrKPIException
 * if an error occurs while computing the result.
 */
public IlrKPIResult getKPIResult() throws IlrKPIException {
  IlrKPIResultMap returnedValue = new IlrKPIResultMap();
  returnedValue.setKPIClassName(PaymentKPIResultAggregator.class.getName());
  Map<String, Serializable> value = new HashMap<String, Serializable>();
  value.put(PaymentKPI.TOTAL_NB_OF_PAYMENTS, this.totalNumberOfPayments);
  value.put(PaymentKPI.NB_OF_MANUAL_PROCESSINGS,
    this.numberOfManualProcessings);
  value.put(PaymentKPI.TOTAL_AMOUNT_MANUALLY_PROCESSED,
    this.totalAmountManuallyProcessed);
  value.put("Average amount of a manually processed payment",
    this.totalAmountManuallyProcessed
    .divide(new BigDecimal(this.numberOfManualProcessings),
    2, BigDecimal.ROUND_HALF_DOWN));
  value.put(PaymentKPI.NB_OF_AUTOMATED_PROCESSINGS,
    this.numberOfAutomatedProcessings);
  value.put(PaymentKPI.TOTAL_AMOUNT_AUTOMATICALLY_PROCESSED,
    this.totalAmountAutomaticallyProcessed);
  value.put("Average amount of an automatically processed payment",
    this.totalAmountAutomaticallyProcessed
    .divide(new BigDecimal(this.numberOfAutomatedProcessings),
    2, BigDecimal.ROUND_HALF_DOWN));
  // … 
  returnedValue.setValue(value);
  return returnedValue;
}

PaymentKPIRenderer implementation

The complete source code of the com.bank.payment.PaymentKPIRenderer class is provided in the sample-src directory of the payment.zip archive provided for download with this article.

The PaymentKPIRenderer implementation renders all entries of the KPI aggregator result in an HTML table. The first column displays the key of the map entry, the second column displays the value, as shown in Listing 12.

Listing 12. rendering KPI in an HTML table
publicvoid encode(IlrKPIResultMap kpiResult, FacesContext context,
  UIComponent component) throws IOException {
 Map<String, Serializable> values = kpiResult.getValue();
  context.getResponseWriter().write("<table>");
  for (String key: values.keySet()) {
    context.getResponseWriter().write("<tr>");
    context.getResponseWriter().write("<td>");
 context.getResponseWriter().write(key);
    context.getResponseWriter().write(":&nbsp;");
    context.getResponseWriter().write("</td>");
    context.getResponseWriter().write("<td>");
    context.getResponseWriter().write(String.valueOf(values.get(key)));
    context.getResponseWriter().write("</td>");
    context.getResponseWriter().write("</tr>");
 }
  context.getResponseWriter().write("</table>");
}

Step 5: deploy and enable the custom scenario suite format

The next step is to package the new scenario suite format in the Decision Center and SSP enterprise application archives. You'll then need to redeploy both enterprise application archives to the application server. Finally, you'll need to enable the new scenario suite format for the payment-processing rule project in the Decision Center.

  1. To package the new scenario suite format in the enterprise application archives, click the Repackage the .ear/.war files in the Actions section of the customization editor, as shown in Figure 20.
    Figure 20. Select Repackage option in the DVS Customization editor
    Select Repackage option in the DVS Customization editor
  2. In the next screen, select to repackage both enterprise application archives and click OK, as shown in Figure 21.
    Figure 21. Repackage SSP and Decision Center
    Repackage SSP and Decision Center
  3. Wait until the repackaging is completed, as indicated by the dialog shown in Figure 22.
    Figure 22. Repackaging status dialog
    Repackaging status dialog
  4. Now that both archives are repackaged, you need to perform one more step before redeploying Decision Center and the SSP to the application server. Because the custom scenario provider accesses the JNDI directory at runtime to retrieve the datasource for the payment database, the deployment descriptor of the SSP must be updated to declare this datasource (named jdbc/lssDB). The descriptor to update is the web.xml file located in the WEB-INF directory of the jrules-ssp-server.war file located in the SSP enterprise archive generated by the repackaging action, as shown in Figure 23.
    Figure 23. Location of the SSP ear in the Payment Simulations project
    Location of the SSP ear in the Payment Simulations project

    In the web.xml file, add the following resources-ref element in the resources references section of the file:

    <resource-ref>
    <res-ref-name>jdbc/lssDB</res-ref-name>
    <res-type>javax.sql.DataSource</res-type>
    <res-auth>Container</res-auth>
    <res-sharing-scope>Unshareable</res-sharing-scope>
    </resource-ref>

    Note that the resources references section of the file is identified in the file by the following header:

    <!-- +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -->
    <!-- R E S O U R C E S -->
    <!-- -->
    <!-- R E F E R E N C E S -->
    <!-- +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -->
  5. After updating the content of the web.xml, you need to define a datasource with the JNDI name jdbc/lssDB in the application server host. For information on how to create a DB2 datasource in WebSphere Application Server V8, refer to the WebSphere Application Server V8 Information Center.
  6. Now you can redeploy the two enterprise applications to the application server. If you need more information on how to redeploy the Decision Center and the SSP, refer to the WebSphere ODM V8 Information Center.
  7. After the redeployment, you need to republish the payment-processing rule project to Decision Center (see this topic in the WebSphere ODM Information Center for more information), and enable the payment database format for simulations on this rule project (more information here), as shown in Figure 24.
    Figure 24. Enabling Payment database simulation format in Decision Center
    nabling Payment database simulation format in Decision Center
  8. To create a simulation using this format, click the Compose tab in Decision Center, select Simulation and click OK. In the third step of the simulation creation, select the Payment database format to display the custom simulation parameters form created by our renderer, as shown in Figure 25.
    Figure 25. Using the payment database format to create a simulation
    Using the payment database format to create a simulation

Simulation results

When we performed the first simulation run with 120 million scenarios, we immediately observed a gradual decline in CPU and network usage, the typical signs that all was not well with the database. We let the simulation run to its conclusion, but the run duration of just over 24 hours was not to our satisfaction. We then printed out the duration of each query to the database and observed that there was, as we suspected, a steady increase in query execution time as the run progressed. After verifying the database configuration parameters, inspecting the DB2 diagnostic messages and explaining our query, we decided to drop an index that we had created on the DATE column. When we did this, the query execution time remained constant and the simulation completed in under two hours! (See Figure 26.) In this case, because the scenarios are read sequentially from a specific starting point, the index is not required and only adds overhead.

There is one minor drawback with removing the date index. When performing smaller simulations (say 1 million scenarios) you may observe that the simulation takes longer to run when using more than one part. The reason is that parts are initialised with the query SELECT ID FROM DATA WHERE DATA.DATE >= {0} ORDER BY ID ASC, which performs better with the date index.

The simulation duration for the first run, after dropping the index on DATE, is within expectations, as shown in Figure 26.

Figure 26. Simulation report for an execution with six parts
Simulation report for an execution with six parts

We used six parts for this run and a cache size of 4000. The parts, which were delimited by dates (in Java long format), were equal in size with 20 million scenarios each, as shown in Table 2.

Table 2. Payment data partitions in the 120M records database
Scenario IDDate (long)
1 1296566652000
20000000 1296666651995
40000000 1296766651995
60000000 1296866651995
80000000 1296966651995
100000000 1297066651995
120000000 1297166651995

The run took 1 hour, 37 minutes and 30 seconds (5850 seconds) to complete.

We then performed another simulation to establish a comparison between the duration of the first run and a simulation that only had one part (one execution thread). The results are shown in Figure 27.

Figure 27. Simulation report for an execution with one part
Simulation report for an execution with one part

The duration of this run, which used only one execution thread, was 3 hours, 5 minutes and 9 seconds (11109 seconds). This is nearly twice as long as the previous run, which had six execution threads. Although the results of comparisons like this may vary depending on the computing resources available for the simulation and the size of the scenario database, the parallel method is clearly advantageous.

Finally, we wanted to get some idea of the impact of the cache size on the simulation duration. We changed the cache size from 4000 to 500 with the results shown in Figure 28.

Figure 28. Simulation report with a cache size of 500
Simulation report with a cache size of 500

As you can see, the difference between the run duration (5861 seconds) compared with that of the first run (5850 seconds) is negligible. Whether you see any variation or not when you change the cache size will depend primarily on your database configuration and network resources.

Additional tests, including varying the number of parts to determine the number that provides the best performance, could also be considered. Because of time constraints, we didn't do any further tests, but it's generally a good practice to carry out sufficient tests to ensure optimum usage of computing resources before making a large-scale simulation capability available to end users.


Conclusion

In this article we introduced the new parallel simulation capability in WebSphere Operational Decision Management V8. We offered insight into how this feature works and discussed how to create the custom artifacts that are required to perform large-scale simulations adapted to your company's needs.

At a high level, we proposed a methodology to provide guidance on how to approach generic large-scale simulation problems. Using a detailed and comprehensive example, we also looked at the specific low-level details of how to implement and deploy the custom artifacts required by the API. In addition, we touched on some of the challenges of working with very large scenario databases. In the example, we showed the impact that less than optimal database design and tuning can have when, by dropping a redundant index, we reduced the simulation time from 24 to less than 2 hours. Finally, by performing simulations using our example project and a database of 120 million scenarios, we demonstrated the benefit of running the simulation in parallel relative to single thread execution by halving the simulation time.


Download

DescriptionNameSize
Sample filespayment.zip40KB

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Business process management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Business process management, WebSphere
ArticleID=836855
ArticleTitle=Running large-scale simulations with WebSphere Operational Decision Management V8
publish-date=09262012