Real-time Surveillance Video Salient Object Detection Using Collaborative Cloud-Edge Deep Reinforcement Learning

In recent years, with the advancement of cloud computing technology and the availability of cheaper hardware, surveillance systems have become more and more common. Unfortunately, most existing systems still face many limitations, such as latency and real-time analysis issues, etc. Edge computing effectively expands the boundaries of cloud computing, migrating some computing and analysis tasks to the edge devices for execution. Edge device could perform video analysis, which may be a good solution. In this paper, we adopt the collaborative Cloud-Edge architecture to analyze surveillance video and extract video keyframes for compressing video data at the edge. Then, we provide a residual U-net neural network to perform salient object detection on the extracted keyframes. Finally, we utilize the deep reinforcement learning Asynchronous Advantage Actor-Critic (A3C) algorithm to perform the residual U-net tasks scheduling, adaptive offloading in the cloud or edge, reducing system latency, and improving real-time performance. We verified the system performance using real road surveillance videos and other public datasets. The experiment results are inspiring. It proves that the real-time processing of the surveillance video system based on a collaborative cloud-edge mechanism could obtain the optimal result within the range of tolerable latency.