计算机科学
公制(单位)
集合(抽象数据类型)
建筑
人工智能
人工神经网络
零(语言学)
机器学习
进化算法
数据挖掘
工程类
哲学
艺术
视觉艺术
语言学
程序设计语言
运营管理
作者
Jianwei Zhang,Lei Zhang,Yan Wang,Junyou Wang,Xin Wei,Wenjie Liu
标识
DOI:10.1142/s0129065723500168
摘要
Neural Architecture Search (NAS) has recently shown a powerful ability to engineer networks automatically on various tasks. Most current approaches navigate the search direction with the validation performance-based architecture evaluation methodology, which estimates an architecture’s quality by training and validating on a specific large dataset. However, for small-scale datasets, the model’s performance on the validation set cannot precisely estimate that on the test set. The imprecise architecture evaluation can mislead the search to sub-optima. To address the above problem, we propose an efficient multi-objective evolutionary zero-shot NAS framework by evaluating architectures with zero-cost metrics, which can be calculated with randomly initialized models in a training-free manner. Specifically, a general zero-cost metric design principle is proposed to unify the current metrics and help develop several new metrics. Then, we offer an efficient computational method for multi-zero-cost metrics by calculating them in one forward and backward pass. Finally, comprehensive experiments have been conducted on NAS-Bench-201 and MedMNIST. The results have shown that the proposed method can achieve sufficiently accurate, high-throughput performance on MedMNIST and 20[Formula: see text]faster than the previous best method.
科研通智能强力驱动
Strongly Powered by AbleSci AI