Deep learning augmented infrared thermography for unmanned aerial vehicles structural health monitoring of roadways

Sinkholes form when culverts beneath roadways fail, posing a safety hazard that requires costly repairs and causes traffic delays. Detecting the onset of sub-pavement voids can prevent sudden pavement failure. This research presents an automated approach for increasing the detection accuracy of sub-pavement voids from infrared (IR) images collected with an unmanned aerial vehicle. A framework is developed that relies on i) principal component thermography analysis to increase accuracy of the damage detection and ii) the EfficientDet algorithm to automate the inspection process. Field tests on a roadway demonstrated the capabilities of the proposed framework in detecting sub-pavements voids with accuracy comparable to ground penetrating radar while significantly reducing the inspection time. The automated detection method yielded a mean average precision of 0.85 when detecting both visible and sub-pavement defects, outperforming other deep learning algorithms by 24%. The feasibility of the approach, best practices, and lessons learned are presented.