Rethinking the performance comparison between SNNS and ANNS

计算机科学 人工智能 人工神经网络 机器学习 模式识别(心理学)
作者
Lei Deng,Yujie Wu,Xing Hu,Ling Liang,Yufei Ding,Guoqi Li,Guangshe Zhao,Peng Li,Yuan Xie
出处
期刊:Neural Networks [Elsevier BV]
卷期号:121: 294-307 被引量:256
标识
DOI:10.1016/j.neunet.2019.09.005
摘要

Artificial neural networks (ANNs), a popular path towards artificial intelligence, have experienced remarkable success via mature models, various benchmarks, open-source datasets, and powerful computing platforms. Spiking neural networks (SNNs), a category of promising models to mimic the neuronal dynamics of the brain, have gained much attention for brain inspired computing and been widely deployed on neuromorphic devices. However, for a long time, there are ongoing debates and skepticisms about the value of SNNs in practical applications. Except for the low power attribute benefit from the spike-driven processing, SNNs usually perform worse than ANNs especially in terms of the application accuracy. Recently, researchers attempt to address this issue by borrowing learning methodologies from ANNs, such as backpropagation, to train high-accuracy SNN models. The rapid progress in this domain continuously produces amazing results with ever-increasing network size, whose growing path seems similar to the development of deep learning. Although these ways endow SNNs the capability to approach the accuracy of ANNs, the natural superiorities of SNNs and the way to outperform ANNs are potentially lost due to the use of ANN-oriented workloads and simplistic evaluation metrics. In this paper, we take the visual recognition task as a case study to answer the questions of "what workloads are ideal for SNNs and how to evaluate SNNs makes sense". We design a series of contrast tests using different types of datasets (ANN-oriented and SNN-oriented), diverse processing models, signal conversion methods, and learning algorithms. We propose comprehensive metrics on the application accuracy and the cost of memory & compute to evaluate these models, and conduct extensive experiments. We evidence the fact that on ANN-oriented workloads, SNNs fail to beat their ANN counterparts; while on SNN-oriented workloads, SNNs can fully perform better. We further demonstrate that in SNNs there exists a trade-off between the application accuracy and the execution cost, which will be affected by the simulation time window and firing threshold. Based on these abundant analyses, we recommend the most suitable model for each scenario. To the best of our knowledge, this is the first work using systematical comparisons to explicitly reveal that the straightforward workload porting from ANNs to SNNs is unwise although many works are doing so and a comprehensive evaluation indeed matters. Finally, we highlight the urgent need to build a benchmarking framework for SNNs with broader tasks, datasets, and metrics.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
更新
PDF的下载单位、IP信息已删除 (2025-6-4)

科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
刚刚
清脆南蕾完成签到,获得积分10
刚刚
medzhou完成签到,获得积分10
刚刚
晨曦完成签到,获得积分10
1秒前
哈哈发布了新的文献求助10
1秒前
Salvator完成签到,获得积分10
1秒前
鲨鱼辣椒完成签到,获得积分10
2秒前
今天开心吗完成签到 ,获得积分10
2秒前
2秒前
小一完成签到,获得积分10
3秒前
Kiosta发布了新的文献求助10
4秒前
lisaltp完成签到,获得积分10
4秒前
静静等待完成签到,获得积分10
4秒前
圆圆发布了新的文献求助10
5秒前
5秒前
等待蚂蚁完成签到 ,获得积分10
6秒前
脑洞疼应助ceeray23采纳,获得20
6秒前
7秒前
kehan完成签到,获得积分10
7秒前
Manchester完成签到,获得积分10
7秒前
xwl9955完成签到 ,获得积分10
7秒前
Slide完成签到 ,获得积分20
7秒前
玛卡巴卡发布了新的文献求助10
7秒前
DarrenVan完成签到,获得积分10
8秒前
大胆的钢笔完成签到,获得积分10
8秒前
江山完成签到,获得积分20
8秒前
8秒前
奈芙莲完成签到,获得积分10
8秒前
Fashioner8351完成签到,获得积分10
9秒前
logan完成签到,获得积分10
10秒前
10秒前
殊量完成签到,获得积分10
10秒前
刘小源完成签到 ,获得积分10
11秒前
科研搬运工完成签到,获得积分10
11秒前
周新运完成签到,获得积分10
11秒前
ying发布了新的文献求助10
11秒前
大山完成签到,获得积分10
11秒前
彳亍完成签到,获得积分10
12秒前
蓝天白云发布了新的文献求助10
13秒前
壮观复天完成签到 ,获得积分10
13秒前
高分求助中
【请各位用户详细阅读此贴后再求助】科研通的精品贴汇总(请勿应助) 10000
The Mother of All Tableaux: Order, Equivalence, and Geometry in the Large-scale Structure of Optimality Theory 3000
International Code of Nomenclature for algae, fungi, and plants (Madrid Code) (Regnum Vegetabile) 500
Maritime Applications of Prolonged Casualty Care: Drowning and Hypothermia on an Amphibious Warship 500
Comparison analysis of Apple face ID in iPad Pro 13” with first use of metasurfaces for diffraction vs. iPhone 16 Pro 500
Towards a $2B optical metasurfaces opportunity by 2029: a cornerstone for augmented reality, an incremental innovation for imaging (YINTR24441) 500
Materials for Green Hydrogen Production 2026-2036: Technologies, Players, Forecasts 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 工程类 有机化学 生物化学 物理 内科学 纳米技术 计算机科学 化学工程 复合材料 遗传学 基因 物理化学 催化作用 冶金 细胞生物学 免疫学
热门帖子
关注 科研通微信公众号,转发送积分 4061661
求助须知:如何正确求助?哪些是违规求助? 3600275
关于积分的说明 11433299
捐赠科研通 3323815
什么是DOI,文献DOI怎么找? 1827483
邀请新用户注册赠送积分活动 897954
科研通“疑难数据库(出版商)”最低求助积分说明 818774