An Evolutionary Framework for Multi-Objective Neural Architecture Search

Neural architecture search (NAS) has garnered significant attention and achieved remarkable success in deep learning. In recent years, the increasing demand for generic, resource-available, and trustworthy AI has shifted research focus beyond mere accuracy to model complexity, energy efficiency, and inference latency, leading to the emergence of multi-objective neural architecture search problems (MONASPs). However, existing methods often integrate classical multi-objective evolutionary algorithms (MOEAs) as the search strategy with problem-specific strategies, limiting their applicability across different deep neural network models and deep learning tasks. Therefore, there is a growing need for MOEA-based search strategies that can accommodate the characteristics of MONASPs. This work proposes an evolutionary framework as a search strategy for MONASPs. Specifically, a coevolutionary mechanism is proposed where an auxiliary population prioritizes decision space diversity to improve the search for multi-modal landscapes. Moreover, a two-stage offspring generation mechanism is devised. The first stage exploits the advantage of a crossover operator to enhance convergence towards the optimal regions in the multi-modal landscape. Then, the second stage adopts a differential evolution operator to manage potential linkages between decision variables, promoting the exploration of diverse architectures along the Pareto front. The effectiveness of this new framework is validated through comparisons with eight representative MOEAs and 22 advanced NAS methods. Experiments are conducted across seven search spaces, utilizing two large common datasets, and assessed by four performance indicators. The results demonstrate the competitiveness and superiority of the framework in providing practitioners with diverse and high-performance architectures.