Leveraging Large Language Models for Voice-Activated Robotic Teleoperation in Construction

Recent advances in construction robotics offer solutions to labor shortages and productivity challenges. One promising approach is teleoperation, where human workers supervise robotic operations. However, many workers lack the technical knowledge to operate robots effectively. To bridge this gap, this study proposes a voice-activated robotic teleoperation framework that enables workers to control robots using natural, everyday language. The system processes voice input through modules for denoising, speaker diarization, and automatic speech recognition (ASR), converting speech into text. A large language model (LLM) then interprets the text to generate both high-level commands (e.g., pick up the bricks) and low-level motor controls (e.g., adjusting joint angles). A case study in bricklaying demonstrates the framework’s potential to improve human–robot interaction on construction sites by making robotic control more intuitive and accessible for non-technical users. This research supports the broader adoption of robotics in construction by lowering the barrier to entry-level construction workers.