A dual-stage system for real-time license plate detection and recognition on mobile security robots

Abstract Automatic license plate recognition (ALPR) systems are increasingly used to solve issues related to surveillance and security. However, these systems assume constrained recognition scenarios, thereby restricting their practical use. Therefore, we address in this article the challenge of recognizing vehicle license plates (LPs) from the video feeds of a mobile security robot by proposing an efficient two-stage ALPR system. Our ALPR system combines the on-the-shelf YOLOv7x model with a novel LP recognition model, called vision transformer-based LP recognizer (ViTLPR). ViTLPR is based on the self-attention mechanism to read character sequences on LPs. To ease the deployment of our ALPR system on mobile security robots and improve its inference speed, we also propose an optimization strategy. As an additional contribution, we provide an ALPR dataset, named PGTLP-v2, collected from surveillance robots patrolling several plants. The PGTLP-v2 dataset has multiple features to cover chiefly the in-the-wild scenario. To evaluate the effectiveness of our ALPR system, experiments are carried out on the PGTLP-v2 dataset and five benchmark ALPR datasets collected from different countries. Extensive experiments demonstrate that our proposed ALPR system outperforms state-of-the-art baselines.