MOFS-REPLS: A large-scale multi-objective feature selection algorithm based on real-valued encoding and preference leadership strategy

Multi-objective feature selection (MOFS) has emerged as a crucial step in constructing efficient machine-learning models. While multi-objective evolutionary algorithms often yield satisfactory sub-optimal solutions, enhancing these algorithms' global optimization capacity remains a central challenge in the field of engineering optimization. To improve the quality of solutions to problems, there is an imperative need for an algorithm with superior optimization capability. This study introduces a large-scale MOFS algorithm based on real-valued encoding and a preference leadership strategy, named MOFS-REPLS, which aims to address the challenge of large-scale sparse feature selection (FS). First, we propose a novel encoding scheme to facilitate broader population exploration. During the population initialization phase, we integrate a ReliefF-guided approach with roulette wheel selection to create the initial population. Second, we introduce a preference leadership strategy that directs individuals toward their respective areas in the Pareto front. Finally, we devise an adaptive learning strategy incorporating ReliefF-guided methods to steer the evolution of the population, thereby mitigating performance deficiencies due to the algorithm's lack of prior knowledge. MOFS-REPLS employs a dual-archive mechanism to maintain diversity within the algorithm and to preserve non-dominated solutions for further exploration. Through experimental assessment using 20 UCI datasets and 10 state-of-the-art algorithms, we demonstrate the effectiveness of MOFS-REPLS. The results show that our proposed algorithm not only maintains high accuracy but also selects a smaller, more relevant set of features, significantly outperforming other FS algorithms in comparison.