A discrete mixture-based kernel for SVMs: Application to spam and image categorization

In this paper, we investigate the problem of training support vector machines (SVMs) on count data. Multinomial Dirichlet mixture models allow us to model efficiently count data. On the other hand, SVMs permit good discrimination. We propose, then, a hybrid model that appropriately combines their advantages. Finite mixture models are introduced, as an SVM kernel, to incorporate prior knowledge about the nature of data involved in the problem at hand. For the learning of our mixture model, we propose a deterministic annealing component-wise EM algorithm mixed with a minimum description length type criterion. In the context of this model, we compare different kernels. Through some applications involving spam and image database categorization, we find that our data-driven kernel performs better.