计算机科学
循环神经网络
人工智能
快照(计算机存储)
深度学习
卷积神经网络
对抗制
压缩传感
计算机视觉
帧(网络)
人工神经网络
模式识别(心理学)
算法
电信
操作系统
作者
Ziheng Cheng,Ruiying Lu,Zhengjue Wang,Hao Zhang,Bo Chen,Ziyi Meng,Xin Yuan
标识
DOI:10.1007/978-3-030-58586-0_16
摘要
We consider the problem of video snapshot compressive imaging (SCI), where multiple high-speed frames are coded by different masks and then summed to a single measurement. This measurement and the modulation masks are fed into our Recurrent Neural Network (RNN) to reconstruct the desired high-speed frames. Our end-to-end sampling and reconstruction system is dubbed BIdirectional Recurrent Neural networks with Adversarial Training (BIRNAT). To our best knowledge, this is the first time that recurrent networks are employed to SCI problem. Our proposed BIRNAT outperforms other deep learning based algorithms and the state-of-the-art optimization based algorithm, DeSCI, through exploiting the underlying correlation of sequential video frames. BIRNAT employs a deep convolutional neural network with Resblock and feature map self-attention to reconstruct the first frame, based on which bidirectional RNN is utilized to reconstruct the following frames in a sequential manner. To improve the quality of the reconstructed video, BIRNAT is further equipped with the adversarial training besides the mean square error loss. Extensive results on both simulation and real data (from two SCI cameras) demonstrate the superior performance of our BIRNAT system. The codes are available at https://github.com/BoChenGroup/BIRNAT .
科研通智能强力驱动
Strongly Powered by AbleSci AI