A relevant feature combination assisted bi-level evolutionary multi-objective algorithm for feature selection in classification

Minimizing the number of selected features and reducing the error rate are the two primary objectives in feature selection problems. However, with the exponentially expanded search space caused by the increasing number of features, existing multi-objective feature selection methods often get trapped in local optima and fail to achieve a good balance between these two objectives. To address these issues, this study introduces definitions of relevant and irrelevant feature combinations to distinguish promising from unpromising feature subsets. These definitions facilitate exploration of the decision space and enable effective approximation to regions where segments of the Pareto front (PF) may reside. By analyzing the relationship between the two objectives, a bi-level environmental selection method is proposed to achieve two goals: (1) ensuring basic convergence performance in terms of error rate and (2) maintaining a sound balance between the two objectives. Subsequently, a novel multi-objective evolutionary feature selection algorithm named DRF-FM is developed based on the aforementioned findings. Extensive experiments were conducted on 22 datasets using five comparative algorithms to verify the performance of DRF-FM. The results demonstrate that DRF-FM outperforms the competitors with the most superior overall performance.