Towards understanding the influence of SVM hyperparameters

Abstract

We investigate the relationship between SVM hyperparameters for linear and RBF kernels and classification accuracy. The process of finding SVM hyperparameters usually involves a gridsearch, which is both time-consuming and resource-intensive. On large datasets, 10-fold cross-validation grid searches can become intractable without supercomputers or high performance computing clusters. We present theoretical and empirical arguments as to how SVM hyperparameters scale with N, the amount of learning data. By using these arguments, we present a simple algorithm for finding approximate hyperparameters on a reduced dataset, followed by a focused line search on the full dataset. Using this algorithm gives comparable results to performing a grid search on complete datasets.