Detection and classification of road signs in raining condition with limited dataset

Abstract The road sign recognition (RSR) system is used to complete two tasks: localizing the traffic sign in an image and then classifying it according to the image features. Some of the applications of such system are incorporated into the Advanced Driver Assistance System (ADAS) and autonomous vehicles. However, the accuracy of the model decreases when changes in lighting or weather occurs, and the lack of training samples taken in rainy condition causes the model to be sub-optimal. The research was conducted with the focus on solving two problems, the accuracy of an object detection model decreases when changes in lighting or weather occurs and the lack of training data, especially images taken under adverse condition. In this work, we compare and analyze three methods; automatic white balance (AWB), policy augmentations and Image-to-Image-translation (I2IT) technique on their performance to detect traffic signs in raining conditions. All methods were built upon the pre-trained SSD-MobileNetV2 model and using the TensorFlow2 framework. The images from the Malaysia Traffic Sign Dataset (MTSD) are used to train all models. Finally based on the result of the three model a final combination model is proposed that achieved the best performance in rainy condition. Experimental results showed that AWB was not that effective in detecting road sign in raining condition, while the other two techniques were highly effective. The final proposed model was implemented by combining policy augmentation and I2IT, obtained an mAP of 0.7967 in clear images and mAP of 0.7160 when rainy images was added to the testing dataset. These corresponds to mAP at 50% IoU of 0.8921mAP@0.5 during clear weather and 0.8340mAP@0.5 in raining images, which outperformed other models. Thus, a road sign detection and classification system that can perform well in rainy condition with limited training dataset has been successfully developed.