计算机科学
尖峰神经网络
人工智能
事件(粒子物理)
模式识别(心理学)
人工神经网络
计算机视觉
Spike(软件开发)
事件数据
数据建模
钥匙(锁)
算法
特征提取
作者
Qirui Huang,J Li,Chenchen Yan,Pengjiang Qian,Guoqi Li,Jiangbo Qian
标识
DOI:10.1109/tcsvt.2026.3717439
摘要
Event data exhibit high temporal resolution, low latency, and spatial sparsity characteristics that are inherently compatible with the energy-efficient computation advantages of spiking neural networks through their unique spike-driven mechanisms. Existing methods typically first convert event streams into frames and then extract features using convolutional operations. However, in the conversion process, a significant amount of fine-grained temporal information is discarded from the event stream, which hinders recognition capability. To address this issue, we propose a frame-point spiking transformer (FPSformer), which introduces event points containing fine-grained temporal information into event frames and employs a spiking cross-attention mechanism to fuse these features, thereby enhancing performance. To further exploit the temporal information in event data and improve the performance of FPSformer, we investigate the distinct characteristics of event data and propose a novel event data pre-training method called the masked autoencoder for event data (EVMAE). While conventional masked autoencoders focus on learning spatial information by reconstructing the original input image rather than learning temporal information, our EVMAE reconstructs finely sliced event frames from coarsely sliced inputs, encouraging the model to focus more on learning temporal representations that are crucial for event-based action recognition. To our knowledge, EVMAE is the first masked modeling approach specifically designed for event-based action recognition. We conduct extensive experiments on multiple neuromorphic datasets to validate the effectiveness of our approach. Compared with existing methods, EVMAE demonstrates superior performance on several downstream tasks.
科研通智能强力驱动
Strongly Powered by AbleSci AI