3D vision-based bolt loosening assessment using photogrammetry, deep neural networks, and 3D point-cloud processing

Structural bolts are essential structural elements. Detection of structural bolt loosening is of great importance to provide earlier warnings of structural damages and prevent catastrophic system-level collapse. Most existing studies about bolt loosening assessment were built in 2D computer vision, where the assessment may be restricted based on the camera views. In this paper, a novel 3D vision-based methodology is proposed for autonomous bolt loosening assessment. First, a 3D point cloud of bolted connection is created using readily available 2D images. Second, a new convolutional neural network (CNN)-based method is developed to localize structural bolts in the 3D point cloud. Further, a 3D point cloud processing algorithm is developed to recognize and quantify bolt loosening. Parameter studies were conducted to investigate the effectiveness of the proposed pipeline. Finally, a real-world implementation has been conducted to quantify bolt loosening on a steel column base connection with bolts. The results indicate that the proposed bolt loosening assessment methodology can effectively localize and quantify bolt loosening at high accuracy and low cost.