A Filter Approach to Multiple Feature Construction for Symbolic Learning Classifiers Using Genetic Programming

Feature construction is an effort to transform the input space of classification problems in order to improve the classification performance. Feature construction is particularly important for classifier inducers that cannot transform their input space intrinsically. This paper proposes GPMFC, a multiple-feature construction system for classification problems using genetic programming (GP). This paper takes a nonwrapper approach by introducing a filter-based measure of goodness for constructed features. The constructed, high-level features are functions of original input features. These functions are evolved by GP using an entropy-based fitness function that maximizes the purity of class intervals. A decomposable objective function is proposed so that the system is able to construct multiple high-level features for each problem. The constructed features are used to transform the original input space to a new space with better separability. Extensive experiments are conducted on a number of benchmark problems and symbolic learning classifiers. The results show that, in most cases, the new approach is highly effective in increasing the classification performance in rule-based and decision tree classifiers. The constructed features help improve the learning performance of symbolic learners. The constructed features, however, may lack intelligibility.