Turning majority voting classifiers into a single decision tree

This paper addresses the issues of intelligibility, classification speed, and required space in majority voting classifiers. Methods that classify unknown cases using multiple classifiers (e.g. bagging, boosting) have been actively studied in recent years. Since these methods classify a case by taking majority voting over the classifiers, the reasons behind the decision cannot be described in a logical form. Moreover, a large number of classifiers is needed to significantly improve the accuracy. This greatly increases the amount of time and space needed in classification. To solve these problems, a method for learning a single decision tree that approximates the majority voting classifiers is proposed in this paper. The proposed method generates if-then rules from each classifier, and then learns a single decision tree from these rules. Experimental results show that the decision trees by our method are considerably compact and have similar accuracy compared to bagging. Moreover, the proposed method is 8 to 24 times faster than bagging in classification.