Graph Neural Network-Based Surrogate Model for Evolutionary Neural Architecture Search

Evolutionary neural architecture search (ENAS) is an approach to automating network architecture design. A primary challenge of ENAS is the demand for expensive computational resources, and many ENAS methods use surrogate models to reduce costs. However, existing architecture representation methods are not rich enough in capturing the information of architectures, making them inadequate for building effective surrogate models. Furthermore, existing research studies often rely on a single model or strategy to predict architecture performance, which is not always accurate or reliable. To alleviate these issues, this article proposes a dual-stage surrogate model based on graph neural networks (GNNs) for ENAS, namely dual-stage surrogate model for evolutionary neural architecture search (DSGENAS). First, to effectively represent the neural architectures for the surrogate model, we employ a GNN-based architecture embedding method to extract architecture features. Second, based on the architecture representation, a dual-stage surrogate model strategy is proposed and integrated into the ENAS framework. This strategy combines two surrogate models in the evolutionary search process, one for global performance-tier learning and the other for local performance relationship learning. Experimental results show that extracting architecture features through GNNs can achieve a more effective architecture representation. The two different surrogate models can jointly assist the search process in ENAS. Furthermore, DSGENAS can achieve accurate and stable prediction results, obtaining state-of-the-art results on neural architecture search (NAS) benchmarks.