Combining computer vision with semantic reasoning for on-site safety management in construction

Computer vision has been utilized to extract safety-related information from images with the advancement of video monitoring systems and deep learning algorithms. However, construction safety management is a knowledge-intensive task; for instance, safety managers rely on safety regulations and their prior knowledge during a jobsite safety inspection. This paper presents a conceptual framework that combines computer vision and ontology techniques to facilitate the management of safety by semantically reasoning hazards and corresponding mitigations. Specifically, computer vision is used to detect visual information from on-site photos while the safety regulatory knowledge is formally represented by ontology and semantic web rule language (SWRL) rules. Hazards and corresponding mitigations can be inferred by comparing extracted visual information from construction images with pre-defined SWRL rules. Finally, the example of falls from height is selected to validate the theoretical and technical feasibility of the developed conceptual framework. Results show that the proposed framework operates similar to the thinking model of safety managers and can facilitate on-site hazard identification and prevention by semantically reasoning hazards from images and listing corresponding mitigations.