A two-step computer vision-based framework for bolt loosening detection and its implementation on a smartphone application

Bolt loosening detection is a labor-intensive and time-consuming process for field engineers. This paper develops a two-step computer vision-based framework to quickly identify bolt loosening angle from field images captured by unmanned aerial vehicle (UAV). In step one, a total of 1200 image samples of bolted structures were used to train faster region based convolutional neural network (Faster R-CNN) for bolt detection from UAV captured images. In step two, computer vision-based technologies, including Gaussian filter, perspective transform, and Hough transform (HT), were performed to quantify bolt loosening angle. The developed framework was then integrated into web server and an iOS application (app) was designed to enable fast data communication between field workplace (UAV captured images) and web server (bolt loosening angle quantification), so that field engineers can quickly view the inspection results on their phone screens. The proposed framework and designed smartphone app greatly help field engineers to improve the accuracy and efficiency for onsite inspection and maintenance of bolted structures.