Automatic recognition of earth rock embankment leakage based on UAV passive infrared thermography and deep learning

Leakage erosion is one of the most harmful driving factors causing river embankment breaches, particularly in flood season. However, manual patrol is the main way to find river embankment leakage presently, which badly hinders disaster prevention. To realize the efficient detection and automatic identification of embankment leakage, for the first time the strategy of UAV carried passive infrared thermography combined with transfer learning is introduced herein as an innovative approach to ensure embankment safety. Especially, the problem of embankment leakage identification is transformed into image classification. The main research objects in this study are slope leakage and piping, two of the most dangerous causes of embankment failure. To obtain sufficient images for model training, an open-air simulation platform which can simulate the slope leakage and piping under the actual service conditions of river embankment is established. A total of more than 500 infrared thermography experiments are conducted on the leakage simulation platform and then an infrared image database containing more than 10,000 images which contain various thermal anomaly areas generated by 6 classes of embankment leakage is established. Using these images and AlexNet-based transfer learning method, an image classification model with excellent performance is trained. This model has a classification accuracy of 94.90%, a small leakage missed rate of 0.64%, and a small false alarm rate of 2.65% on the test set. Moreover, before model deployment, visualization techniques such as t-SNE and Grad-CAM are adopted to provide interior insight of the model to ensure that the objects of concern on which the model makes its classification decisions are reasonable. Finally, field tests demonstrated strong feasibility of UAV carried infrared thermography combined with this well-trained model, and revels that the proposed leakage detection and recognition approach has good applicability and generalization.