Enhancing the Worker–Robot Interaction for Construction Task Operation: A Preliminary Study Using Large Language Model-Based Methods

Advancements in robotics offer a promising solution to improve safety and productivity in the construction industry. Robotic arms are widely used and well-suited for modern construction tasks; however, their operational complexity limits broader adoption in the field. This study develops an LLM-based interface that interprets workers’ natural language commands into arm-based robots actions using customized prompt engineering. The LLM-based interface uses a tree-of-thought (ToT) algorithm to generate accurate waypoints and trajectories for robotic arms based on natural language commands, reducing the need for workers to have prior expertise in robot control. To evaluate the interface, a simulated BrickBot experiment, where participants used the interface to control a UFactory robotic arm for bricklaying tasks using the interface in the simulated environments. The results showed an average of 93.5% control accuracy for two tasks from participants. These findings highlight the potential to improve human–robot interaction and overall operational efficiency of construction tasks.