Pavement pothole detection system based on deep learning and binocular vision

Due to strong vibrations and the impact on high-speed vehicles, road potholes may cause discomfort to passengers, damaging the durability of the suspension and the integrity of the cargo. Therefore, a road pothole detection system is proposed, which uses deep learning and binocular vision for precise detection in the front. 6,848 road surface pothole condition recognition datasets were constructed using a vehicle mounted binocular camera. The group attention shuffle block (GASB) is designed to enhance the expression of channel and spatial feature information in road images, while improving the existing shuffling network (ShuffleNetv2). This establishes a ShuffleNetv2 (GASB-ShuffleNetv2) model based on mixed attention for recognizing the state of road potholes. The experimental results show that the model has better accuracy than the basic model and can effectively detect road potholes. In addition, we replaced the ordinary convolution in the CenterNet feature extraction network with pyramid convolution with multiple receptive fields, and designed a feature fusion module in the same network to fuse low-level and high-level features related to holes, thus establishing a PF-CenterNet that combines pyramid convolution with feature fusion to detect areas containing road potholes. A pothole distance estimation model based on binocular vision was established by analyzing the stereo ranging model and semi-global block matching algorithm. After parameter calibration, images were rectified and stereo-matched to generate a disparity map. This map was optimized using a weighted least squares filter to fill blank areas. The 3D coordinates are then calculated based on the disparity provided with distance information. Finally, vehicle experiments were conducted to verify the effectiveness of the algorithm in meeting detection requirements while considering long-range perception accuracy. The experimental results show that the system can meet the needs of unmanned vehicles, enabling them to perceive potholes in advance, thereby issuing timely warnings to drivers.