Fall-Related Safety Hazard Analysis in Construction Using Vision-Language Models and Natural Language Inference

Fall accidents account for more than one-third of fatalities in construction, and fall-related hazards introduce a significant risk of occupational injuries to construction workers. Although numerous efforts have been devoted to preventing falls, most studies have focused on detecting predefined hazards, with less consideration of understanding the context in which hazardous conditions arise. Most previous studies are limited in identifying combined hazardous conditions, consisting of multiple safety violations, such as a missing safety lanyard on an unprotected open edge on a scaffold. In this context, this study proposes an integrated fall hazard analysis framework using a vision-language model that converts visual information into hazard-descriptions aligned with legal safety standards. Then, the framework analyzes the hazard-related texts using natural language inference to recognize the risk of hazards. By leveraging the BLIP-2 architecture, the proposed method establishes independent image captioning models for three critical categories directly associated with fall accidents: openings, scaffolding, and worker behavior. The framework classifies images as “Safe” or “Risk” based on generated descriptions that strictly adhere to occupational safety and health standards. The proposed fall-hazard assessment framework achieves an F1-score of 0.84, indicating its significant practical potential for comprehensive fall hazard analysis in real-world construction environments.