Evaluating the model
Before you use the model for scoring purposes, you should evaluate how good the model is. The kind of results available for evaluating the model depend on the technique used to generate the model. This example uses the results available with the Propensity to Purchase feature, available in the Direct Marketing add-on option
Propensity to Purchase produces an overall model quality chart and a classification table that can be used to evaluate the model.
The overall model quality chart provides a quick visual indication of the model quality. As a general rule, the overall model quality should be above 0.5.
To confirm that the model is adequate for scoring, you should also examine the classification table.
The classification table compares predicted values of the target field to the actual values of the target field. The overall accuracy rate can provide some indication of how well the model works, but you may be more interested in the percentage of correct predicted positive responses, if the goal is to build a model that will identify the group of contacts likely to yield a positive response rate equal to or greater than the specified minimum positive response rate.
In this example, the classification table is split into a training sample and a testing sample. The training sample is used to build the model. The model is then applied to the testing sample to see how well the model works.
The specified minimum response rate was 0.05 or 5%. The classification table shows that the correct classification rate for positive responses is 7.43% in the training sample and 7.61% in the testing sample. Since the testing sample response rate is greater than 5%, this model should be able to identify a group of contacts likely to yield a response rate greater than 5%.