Learning by Bagging and Adaboost based on Support Vector Machine

Ensemble of classifiers (Multiple classifier system) and Support Vector Machine (SVM) are now well established research lines in machine learning. Recently, some works devoted to SVM-based ensembles report that the most popular ensembles creation methods Bagging and Adaboost are not expected to improve the performance of SVMs and sometimes they even worsen the performance, due to that SVM is stable and strong classifier. In this paper, we focus on adapting Bagging and Adaboost to SVM. The framework of Bagging is extended by introducing the Class-wise expert classifiers, then we proposed the improved algorithm CeBag. The weighting rule of AdaBoost is modified to deal with the overfitting problem which may be even worse when boosting strong classifiers, and the strength of SVM is weakened by adaptively adjusting the kernel parameters, then we proposed the algorithm WwBoost. Experiments implemented on IDA benchmark data sets show that our algorithms are effective in building ensemble of SVMs.