Object Pose Estimation From RGB-D Images With Affordance-Instance Segmentation Constraint for Semantic Robot Manipulation

Object pose estimation is a crucial task for semantic robot manipulation involving the detection of suitable manipulation regions. Given the diversity of object shapes and scene complexities, object pose estimation remains an immense challenge. Accordingly, the letter presents a new approach for object pose estimation from RGB-D images, utilizing the affordance-instance segmentation constraint for semantic robot manipulation. An Object Affordance-Instance Segmentation Network (OAISNet) is designed to improve the segmentation accuracy of both object affordances and object instances. The training of the OAISNet necessitates a substantial quantity of data. A dataset automatic generation method is designed to quickly generate data with multiple labels, reducing the burden of manual annotation. Finally, object affordances are combined with the point pair features to establish affordance-based point pair features for object pose estimation. Experimental results show that the OAISNet improves the performance of object segmentation, and the affordance-based object pose estimation approach improves the accuracy and efficiency of object pose estimation.