Testing in the Business console

The Business console provides capabilities to test sets of rules against usage scenarios.

During rule development, you can create and run test suites in change activities or ungoverned branches in a decision service. You can also run test suites in validation activities to test the content of a release before deployment.

To understand how testing works, consider what you can change in a decision service:
  • The set of rules that you are testing. The set is identified by the decision operation, which defines a subset of rules within the release or change activity on which the testing is being done (see Identifying a set of rules).
  • The scenarios that you are submitting. Each scenario contains all the information that is required to process a transaction. This information can be real or fictitious data. You store this scenario data in a scenario file, which you generate, complete with data, and then upload to Decision Center. You also use this scenario file to store the expected results for each scenario.
  • The test suite that you create and run to provide feedback on the performance of a set of rules without changing the rules or the applications on which they act.

Running a test suite compares the results that you expect to the actual results that are obtained from applying rules to your scenarios. The test generates a report that shows the results for each scenario, and the success rate as the percentage of scenarios that generated their expected results.

Note: If you do not see the option to select an operation when you create a test suite, you can go back to the Test tab and select the operation from the drop-down list in the upper-right corner.

Scenarios

Scenarios represent real or fictitious use cases that you use to validate the behavior of your rules. Each scenario contains all the information that is required for your rules to run properly.

For example, a loan application might require the following information, which must be completed for each scenario:

  • Borrower: Sam Adams

  • Credit Score: 600

  • Yearly Income: 80000

  • Duration: 24 (months)

  • Amount: 100000

  • Yearly Interest Rate: 5 (%)

You can generate scenarios in an Excel format. Each row represents a scenario, and the columns indicate where to put the data for each scenario. For example, the following Scenarios sheet contains four scenarios that validate loan risk from a very low risk loan to a loan where the amount is too high:
Image shows a Scenarios sheet in a scenario file.
Important:

You can also generate scenarios as XML scenario descriptor files. In the Business console, these files are referred to as test data providers, and they are similar to the data providers used for simulation. When you generate a scenario file, in the drop-down list Scenario file format, select Test data provider. You can then select the data that you want to test, provide them with an identifier and a name in the corresponding columns, and select expected execution details. This generates a template data provider, which you can download as a .tstx file. For more information about data providers, see Setting up custom data providers.

Scenario limitations

Limitations apply to generating Excel scenario files for test suites that use the decision engine. If you cannot generate a file, contact your system administrator.

Also, you cannot test a ruleset with more than 255 input parameters. If you reach this limit, the following error appears during the test:

An error has occurred in building a ruleset for executing the test suite.
ilog.rules.teamserver.model.IlrTestingException: There were errors during build: Too many parameters
in com.ibm.rules.generated.EngineDataClass.expectedResult

Follow these steps if you must test more input parameters than the limit:

  1. Create a type with the each parameter as an attribute.
  2. Create a variable of this type.
  3. Create a new ruleflow:
    1. In the start node, initialize the parameters with the fields of this new variable.
    2. Invoke the actual ruleflow to be tested as a subflow.
  4. Create a new operation that uses the new variable as an input/output parameter, and the new ruleflow.
  5. Use the new operation in Decision Center testing.

Test suites

You run test suites to verify that your rules are correctly designed and written. The test suites compare your expected results with the actual results that come from applying your rules against the scenarios that you defined.

You set up the expected results in a separate sheet alongside your Scenarios sheet.

Image shows the Expected Results sheet in a scenario file.
The example tests two different aspects of the loan request:

  • Whether the loan is approved.

  • Which message the loan request application generates.

The Expected Results sheet represents these two tests as columns. You specify the expected results for all the scenarios necessary to cover the validation of your rules.

The test suite returns a report that compares your expected results with the actual results of the execution. Each test in a test suite is successful if all the expected results match the actual results.

You can include an Expected Execution Details sheet for more technical tests, for example, to list the rules that are run or the duration of a run.
Note: When you import a test suite from the release in a validation activity, this test suite is considered as a reference, and is locked in all change activities and validation activities of this release.