EST-NAS: An evolutionary strategy with gradient descent for neural architecture search

计算机科学 初始化 梯度下降 建筑 人工神经网络 进化算法 人工智能 网络体系结构 搜索成本 机器学习 艺术 计算机安全 微观经济学 经济 视觉艺术 程序设计语言
作者
Zicheng Cai,Lei Chen,Shaoda Zeng,Yutao Lai,Hai-Lin Liu
出处
期刊:Applied Soft Computing [Elsevier BV]
卷期号:146: 110624-110624 被引量:4
标识
DOI:10.1016/j.asoc.2023.110624
摘要

Using weight-sharing and continuous relaxation strategies, the gradient descent-based differential architecture search has achieved great success in automatically designing neural network architectures. However, unresolved issues, i.e., the local optimum dilemma of the gradient descent method, and the network performance collapse of the searched architecture with too many unreasonable operations, are still frustrating for researchers and practitioners. To address these two issues, a novel and efficient neural architecture search approach based on a hybrid evolutionary strategy, termed EST-NAS, is proposed in this paper. In particular, we propose using a new evolutionary strategy to explore various search directions based on the gradient descent-based neural network architecture search, aiming at obtaining a more excellent architecture. In the proposed EST-NAS, the gradient descent architecture search is performed first, and then the best architecture obtained is utilized to design an efficient initialization for the following evolutionary strategy-based architecture search. By hybridizing evolutionary strategy with gradient descent-based search, EST-NAS can improve the performance of the searched architecture with better search efficiency. Meanwhile, the validation accuracy is applied to directly measure the importance of operations, which reduces the error in the relationship between operation and task performance. Extensive experimental results in the various datasets on different search spaces show that the proposed EST-NAS achieves remarkably competitive performance with less search cost, compared to other state-of-the-art NAS approaches.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
更新
PDF的下载单位、IP信息已删除 (2025-6-4)

科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
cc发布了新的文献求助20
2秒前
博修发布了新的文献求助10
2秒前
LIUXU完成签到,获得积分10
4秒前
loong完成签到,获得积分10
5秒前
pluto应助ShengzhangLiu采纳,获得10
6秒前
6秒前
苹果发布了新的文献求助10
7秒前
14秒前
我不爱池鱼应助breath采纳,获得10
17秒前
多情凝荷完成签到 ,获得积分10
19秒前
补药完成签到,获得积分10
20秒前
leiiiiiiii完成签到,获得积分10
21秒前
浅色墨水完成签到,获得积分10
21秒前
YamDaamCaa应助wnche采纳,获得260
25秒前
华仔应助补药采纳,获得10
26秒前
MingQue完成签到,获得积分10
31秒前
jenningseastera完成签到,获得积分0
32秒前
36秒前
Ammr完成签到 ,获得积分10
36秒前
37秒前
充电宝应助liu采纳,获得10
37秒前
懵懂的雪糕完成签到 ,获得积分10
37秒前
ljx完成签到 ,获得积分10
39秒前
可爱的函函应助zifeimo采纳,获得10
40秒前
阳光沛柔发布了新的文献求助30
40秒前
赘婿应助踏山河采纳,获得10
43秒前
pluto应助ShengzhangLiu采纳,获得10
44秒前
45秒前
晴空万里完成签到,获得积分10
47秒前
Zsx关闭了Zsx文献求助
48秒前
OGB应助zhhr采纳,获得10
48秒前
49秒前
所所应助自觉竺采纳,获得10
49秒前
50秒前
科研通AI2S应助breath采纳,获得10
50秒前
怕孤单的山河完成签到 ,获得积分10
51秒前
54秒前
55秒前
qks完成签到 ,获得积分0
56秒前
56秒前
高分求助中
(禁止应助)【重要!!请各位详细阅读】【科研通的精品贴汇总】 10000
Semantics for Latin: An Introduction 1099
Biology of the Indian Stingless Bee: Tetragonula iridipennis Smith 1000
Robot-supported joining of reinforcement textiles with one-sided sewing heads 700
Thermal Quadrupoles: Solving the Heat Equation through Integral Transforms 500
SPSS for Windows Step by Step: A Simple Study Guide and Reference, 17.0 Update (10th Edition) 500
PBSM: Predictive Bi-Preference Stable Matching in Spatial Crowdsourcing 300
热门求助领域 (近24小时)
化学 材料科学 医学 生物 工程类 有机化学 生物化学 物理 内科学 纳米技术 计算机科学 化学工程 复合材料 遗传学 基因 物理化学 催化作用 冶金 细胞生物学 免疫学
热门帖子
关注 科研通微信公众号,转发送积分 4126261
求助须知:如何正确求助?哪些是违规求助? 3663827
关于积分的说明 11593227
捐赠科研通 3363465
什么是DOI,文献DOI怎么找? 1848222
邀请新用户注册赠送积分活动 912232
科研通“疑难数据库(出版商)”最低求助积分说明 827935