Surface defect detection of civil structures using images: Review from data perspective

As civil structures age and deteriorate, it becomes crucial to conduct structural health monitoring (SHM) to ensure safety and timely maintenance. Surface defect detection plays a vital role in SHM by providing an initial assessment of structural conditions. Recent advancements in deep learning and automation techniques have led to extensive exploration of deep learning-based surface defect detection using images in the field of civil engineering. However, the performance of surface defect detectors heavily relies on the quality of data used for training, and data-related challenges can significantly impact the practicality of these detectors. These challenges encompass inherent and external negative characteristics of the raw data, including images and labels, which can significantly impact the performance of surface defect detectors. Unfortunately, there is a lack of systematic studies that review and discuss data-related challenges and their solutions in surface defect detection. To bridge this research gap, this study aims to review previous studies on deep learning-based surface defect detection using images, with a specific focus on the data perspective. A total of 237 journal papers were selected and critically analyzed in terms of deep learning tasks and application domains. The study summarizes the data-related challenges that affect the performance and applicability of surface defect detectors, along with the corresponding solutions proposed in the selected papers. While various methods have been proposed to address these challenges, limitations still exist and need to be addressed in future research. To guide future studies in addressing data-related issues, a data management framework is designed, encompassing fixed factors and variable factors derived from the selected papers. Furthermore, the study provides suggestions for future research opportunities to offer insights to researchers in the field of deep learning-based surface defect detection.