MemATr: An Efficient and Lightweight Memory-augmented Transformer for Video Anomaly Detection

Anomaly detection in videos is a long-standing and challenging problem. Previous methods often adopt deep and large neural networks to achieve the best detection accuracy; however, the high computational costs prevent them from being used in real-world applications with constrained computational resources. In this paper, we develop a mem ory- a ugmented tr ansformer named MemATr, which is capable of detecting video anomalies effectively. The proposed network is lightweight and can be easily deployed on mobile devices. Furthermore, we propose a memory transformer module to make predictions that are closer to normal inputs, thereby leading to a higher error for abnormal input patterns. Memory-attention is the main component of the proposed memory transformer, which can retrieve the features from learnable values rather than from the backbone like previous methods. Extensive experiments on the UCSD Ped2, CUHK Avenue, and ShanghaiTech benchmarks can demonstrate that our model has a significantly smaller model size while still achieving competitive detection accuracy. Our model has only 1/12 the number of parameters of the baseline model. Besides, our model achieves a 4.6% increase in accuracy on the ShanghaiTech dataset and has roughly the same accuracy compared with the baseline on the other two datasets. We validate the performance of the proposed model on the mobile device and the result shows it only has 49.8ms latency. The effectiveness of the proposed method on mobile devices is further supported by experimental results. A new quantitative parameter AMD (Applicability for Mobile Devices) is proposed to offer a novel approach to assist in making trade-offs for mobile devices. The proposed model obtains state-of-the-art results in terms of AMD.