Predicted-by-Observed Chart

Figure 1. Predicted-by-observed chart
Predicted-by-observed chart

For categorical dependent variables, the predicted-by-observed chart displays clustered boxplots of predicted pseudo-probabilities for the combined training and testing samples. The x axis corresponds to the observed response categories, and the legend corresponds to predicted categories.

  • The leftmost boxplot shows, for cases that have observed category No, the predicted pseudo-probability of category No. The portion of the boxplot above the 0.5 mark on the y axis represents correct predictions shown in the classification table. The portion below the 0.5 mark represents incorrect predictions. Remember from the classification table that the network is very good at predicting cases with the No category using the 0.5 cutoff, so only a portion of the lower whisker and some outlying cases are misclassified.
  • The next boxplot to the right shows, for cases that have observed category No, the predicted pseudo-probability of category Yes. Since there are only two categories in the target variable, the first two boxplots are symmetrical about the horizontal line at 0.5.
  • The third boxplot shows, for cases that have observed category Yes, the predicted pseudo-probability of category No. It and the last boxplot are symmetrical about the horizontal line at 0.5.
  • The last boxplot shows, for cases that have observed category Yes, the predicted pseudo-probability of category Yes. The portion of the boxplot above the 0.5 mark on the y axis represents correct predictions shown in the classification table. The portion below the 0.5 mark represents incorrect predictions. Remember from the classification table that the network predicts slightly more than half of the cases with the Yes category using the 0.5 cutoff, so a good portion of the box is misclassified.

Looking at the plot, it appears that by lowering the cutoff for classifying a case as Yes from 0.5 to approximately 0.3—this is roughly the value where the top of the second box and the bottom of the fourth box are—you can increase the chance of correctly catching prospective defaulters without losing many potential good customers. That is, moving from 0.5 to 0.3 along the second box incorrectly reclassifies relatively few non-defaulting customers along the whisker as predicted defaulters, while along the fourth box, this move correctly reclassifies many defaulting customers within the box as predicted defaulters.

Next