Cooperative multi-camera vehicle tracking and traffic surveillance with edge artificial intelligence and representation learning

Traffic surveillance cameras are the eyes of the Intelligent Transportation Systems (ITS). However, they are currently isolated and can only extract information from each of their fixed views. To track vehicles across multiple cameras and help public agencies collect link travel time and speed information, an Edge-empowered Cooperative Multi-camera Sensing (ECoMS) System is proposed. ECoMS system presents a novel algorithmic and edge-server cooperative system construct to push edge computing and multi-camera re-identification workflow serving for traffic sensing based on Internet of Things (IoT) architecture. On the algorithm side, ECoMS system proposes a featherlight edge-based computer vision framework for vehicle detection, tracking, and features selection process in a real-time manner. Then, by only sending the objects’ representations to the server, the high-bandwidth data transmission and the heavy post-processing system can be abandoned. Furthermore, a hierarchical clip-based deep vehicle re-identification framework is proposed and integrated into the ECoMS system, and significantly outperforms other state-of-the-art methods by 4%–8% on Rank-1 accuracy. Finally, to balance the accuracy level of different camera pairs, a collaborative cross-camera traffic information estimation framework based on kernel density estimation with kernel smoother is implemented, which can get the precise link and region traffic information together with distributions (less than 1.01 KL distance). By maximizing the cooperation of the computational resources, orchestrating the data transmission and integrating road network graph features in the system, the ECoMS can precisely model the network-scale traffic information in a flexible, cost-effective, and scalable workflow. To the author’s best knowledge, ECoMS is the first multi-camera vehicle tracking and traffic monitoring system based on cooperative IoT architecture.