Kernel Ridge Regression

Kernel Ridge Regression is an extension procedure that uses the Python sklearn.kernel_ridge.KernelRidge class to estimate kernel ridge regression models. Kernel ridge regression models are nonparametric regression models that are capable of modeling linear and nonlinear relationships between predictor variables and outcomes. Results can be highly sensitive to choices of model hyperparameters. Kernel Ridge Regression facilitates choice of hyperparameter values through k-fold cross-validation on specified grids of values using the sklearn.model_selection.GridSearchCV class.

Example
Statistics
Additive_CHI2, CHI2, Cosine, Laplacian, Linear, Polynomial, RBF, Sigmoid, Alpha, Gamma, Coef0, Degree, crossvalidation, observed versus predicted, residuals versus predicted, dual weight coefficients, kernel space weight coefficients.

Data considerations

Data
  • You can specify any or all of the eight different kernel functions.
  • The selected kernel function determines which hyperparameters are active.
  • Hyperparameters include alpha for ridge regularization that are common to all kernels plus as many as three other hyperparameters for each specific kernel function.
  • When multiple kernel subcommands are specified, or more than one value for any parameter is specified, a grid search with cross-validation to evaluate models is performed, and the best fitting model that is based on held out data is selected.
  • The extension accepts split variables from the Split File procedure and weights using the Weight Cases procedure.
  • When weights are included, they are used in creating fitted values in all analyses. Due to limitations in the score method in the sklearn.model_selection.GridSearchCV class, crossvalidation evaluations that are used for model selection are not weighted.
Assumptions

Obtaining a Kernel Ridge Regression

This feature requires SPSS® Statistics Standard Edition or the Advanced Statistics Option.

  1. From the menus choose:

    Analyze > Regression > Kernel Ridge...

  2. Select a Dependent variable.
  3. Select one or more Independent(s) variables.
  4. The default Single model setting is used when only one value for each kernel function parameter is specified. When the Single model setting is selected, you cannot specify additional Kernel(s) functions and weights are fully applied throughout the analysis, evaluation, and scoring of results. You can also use the up and down arrow controls to rearrange the kernel functions.

    Optionally, select Model selection from the Mode list.

    When Model selection is selected from the Mode list, you can add multiple kernel functions to the Kernel(s) list.
    1. Click the add control (+) to include additional kernel functions.
    2. Click the empty cell in the Kernel column to select a kernel function.
    3. Double-click any kernel function row cell to specify kernel function parameter values for the corresponding column (Alpha, Gamma, Coef0, Degree). For more information, see Kernel Parameters. The default kernel function tuning parameters are listed below.
      Additive_CHI2
      ALPHA=1 GAMMA=1
      CHI2
      ALPHA=1 GAMMA=1
      Cosine
      ALPHA=1
      Laplacian
      ALPHA=1 GAMMA=1/p
      Linear
      The default kernel function. ALPHA=1
      Polynomial
      ALPHA=1 GAMMA=1/p COEF0=1 DEGREE=3
      RBF
      ALPHA=1 GAMMA=1/p
      Sigmoid
      ALPHA=1 GAMMA=1/p COEF0=1
    Note: When more than one value for any kernel function parameter is specified, a grid search with cross-validation to evaluate models is performed, and the best fitting model that is based on held out data is selected.
  5. Optionally, click Options to specify the number of crossvalidation folds, display options, plot settings, and items to save. For more information, see Kernel Ridge Regression: Options.
  6. Click OK.

This procedure pastes KRR command syntax.