Asphalt pavement fatigue crack severity classification by infrared thermography and deep learning

Fatigue cracking is usually associated with the structural failure of asphalt pavement. This work aims to apply infrared thermography and deep learning, especially convolutional neural network (CNN), for asphalt pavement fatigue crack severity classification. A dataset of asphalt pavement fatigue cracking was built with four severity levels (no, low-severity, medium-severity, and high-severity) and three image types (visible, infrared, and fusion; fusion is the fusion of visible and infrared images). Thirteen CNN models were trained and evaluated based on accuracy, complexity (computational and model), and memory usage. This work applied Grad-CAM and Guided Grad-CAM to interpret the CNN model for classifying the different severity levels of different crack types (fatigue and longitudinal or transverse cracks) for the three image types. This work investigated the effect of image types on classifying different severity levels of fatigue crack and discussed the applicability of infrared thermography on crack detection (crack severity classification and crack segmentation). The results show that the CNN model had the highest accuracy on the infrared image, followed by the fusion image, and the lowest on the visible image. EfficientNet-B4 achieved the highest accuracy on all three image types, while the accuracy of CNN exceeded 0.95 on all three image types. Different image types made the CNN model have different accuracy in classifying the different severity levels of different crack types, which was interpreted by Grad-CAM and Guided Grad-CAM. Based on the high accuracy and reliability, the fusion image could be an accurate, efficient, and reliable method for crack detection.