VC-based confidence and credibility for support vector machines

Abstract

Assigning a confidence and a credibility measures is a challenging stochastic inference problem. Some algorithms only yield the predicted value without evaluating the measure of confidence or credibility over the decision. Support vector machines (SVM) is one algorithm that showed state-of-the-art decision accuracy but lacks a measure of confidence and credibility over the decisions. In this paper we propose a new confidence measure based on the Vapnik and Chervonenkis (VC) dimension of a learning algorithm and the notion of complexity as defined by Kolmogorov. We also propose a new credibility measure based on the VC dimension. The resulting confidence and credibility measures are then tested on the well-known US postal handwritten digit recognition, on the Wisconsin breast cancer dataset and are also tested for agitation detection. The results show high and improved correlation between the decision and the confidence/credibility measures compared to Vovk’s and Platt’s methods. © 2014, Springer-Verlag Berlin Heidelberg.