Vision-based excavator pose estimation for automatic control

Hydraulic excavators, widely employed in harsh environments, have garnered increased attention in recent years from manufacturers and researchers for automatic operation, particularly through vision-based manipulator pose measurement. This paper utilized a mixed review method to explore the vision-based measurements as well as other applications. A quantitative analysis of computer vision (CV) applications on excavators was performed through a literature search, leading to the classification of these applications into three categories based on keywords. Subsequently, a comprehensive literature review identified four key requirements and challenges for vision-based measurement in the context of automatic control, followed by a discussion on the future prospects of marker-based and deep learning-based manipulator pose measurement for automatic control. This paper offers an in-depth examination of CV applications in excavators, emphasizing the pertinent challenges and forthcoming trends in vision-based manipulator pose measurement for automatic control.