Integration of Vision-Based Object Detection and Grasping for Articulated Manipulator in Lunar Conditions

Robotic manipulators will play a crucial role in performing various tasks on the Moon, such as resource extraction, construction and assembly of human outposts. To enable autonomy lunar robotic applications, robust vision-based frameworks must be integrated efficiently. However, this encounters numerous challenges, for instance, uneven terrain configurations and extreme lighting conditions on the Moon. This paper presents a versatile task pipeline that incorporates object detection, instance segmentation and grasp detection, the results of which can be applied to diverse applications of manipulators. We demonstrate the successful execution of two experiments by a 7-DoF manipulator. The first experiment involves stacking differently sized rocks on a non-flat surface in challenging lighting conditions with an impressive success rate of 92%. In the second experiment, we assemble 3D-printed robotic components, thus paving the path to initiate more complex tasks in the future.