Fine-Tuning Vision Transformer (ViT) to Classify Highway Construction Workers’ Activities

Accurate construction workers' activity detection distinguishes workers' activity efficiency and identifies activities with high risks, which enhances construction productivity and safety management. Automated construction workers' activity has become feasible in the construction industry since the emerging of computer vision techniques. However, few studies explored workers' activity detection in transportation-related work zones (e.g., mobile work zone operations), which have their unique characteristics and different requirements than building construction jobsites. Previous studies in the construction domain usually used convolutional neural networks (CNN) for computer vision-related tasks. The transformer-based model achieved higher performance in computer vision tasks after it was first applied in 2020. However, few studies have applied the transformer model for worker activity identification. Therefore, this study aims to detect construction workers' activities in mobile work zones using a pre-trained Vision Transformer (ViT) model. This study starts with the video data collection of construction workers. Then, a dataset containing different activities is developed by manual labeling. Next, the ViT model is fine-tuned using the dataset developed in this study. The results show the model has 94.17% overall accuracy and achieves 100%, 100%, and 84% precision in detecting "placing mix," "shoveling," and "walking," respectively, and outperforms a CNN-based classification model.