Change in Deviance Versus Predicted Probabilities

The change in deviance is not an option in the Save dialog box, but it can be estimated by squaring the studentized residuals.

  1. To create the change in deviance, from the menus choose:

    Transform > Compute Variable...

    Figure 1. Compute Variable dialog box
    Compute Variable dialog box
  2. Type chgdev as the Target Variable.
  3. Type sre_1**2 as the Numeric Expression.
  4. Click OK.

    The squared studentized residuals have been saved to chgdev.

  5. To produce the residual plot, from the menus choose:

    Graphs > Chart Builder...

    Figure 2. Chart Builder
    Chart Builder
  6. Select the Scatter/Dot gallery and choose Simple Scatter.
  7. Select chgdev as the variable to plot on the Y axis.
  8. Select Predicted probability as the variable to plot on the X axis.
  9. Click OK.
Figure 3. Change in deviance (squared studentized residuals) vs. predicted probabilities
Change in deviance (squared studentized residuals) vs. predicted probabilities

The change in deviance plot helps you to identify cases that are poorly fit by the model. Larger changes in deviance indicate poorer fits. There are two distinct patterns in the plot: a curve that extends from the lower left to the upper right, and a curve that extends from the upper left to the lower right.

  • The curve that extends from the lower left to the upper right corresponds to cases in which the dependent variable has a value of 0. Thus, non-defaulters who have large model-predicted probabilities of default are poorly fit by the model.
  • The curve that extends from the upper left to the lower right corresponds to cases in which the dependent variable has a value of 1. Thus, defaulters who have small model-predicted probabilities of default are poorly fit by the model.

By identifying the cases that are poorly fit by the model, you can focus on how those customers are different, and hopefully discover another predictor that will improve the model.

Next