Tuning an SVM Model

Besides the separating line between the categories, a classification SVM model also finds marginal lines that define the space between the two categories.

Figure 1. Data with a preliminary model
Data with a preliminary model

The data points that lie on the margins are known as the support vectors.

The wider the margin between the two categories, the better the model will be at predicting the category for new records. In the previous example, the margin is not very wide, and the model is said to be overfitted. A small amount of misclassification can be accepted in order to widen the margin; an example of this is shown in the following figure.

Figure 2. Data with an improved model
Data with an improved model

In some cases, linear separation is more difficult; an example of this is shown in the following figure.

Figure 3. A problem for linear separation
A problem for linear separation

In a case like this, the goal is to find the optimum balance between a wide margin and a small number of misclassified data points. The kernel function has a regularization parameter (known as C) which controls the trade-off between these two values. You will probably need to experiment with different values of this and other kernel parameters in order to find the best model.