Robot Grasping through a Joint-Initiative Supervised Autonomy Framework

Robot grasping applications are faced with challenges and limitations leading to errors that affect their performance and accuracy. Although these errors are reduced in expensive industrial systems, low-cost robots are still prone to inaccurate perception and execution due to their limited hardware and software capabilities. To mitigate these challenges and limitations, this work develops a Joint-Initiative Supervised Autonomy (JISA) framework for robot grasping. In the proposed system, a human supervisor provides assistance to the robot's perception and planning modules, where the assistance is triggered by requests made by the robot based on its self confidence (SC) metric. Moreover, the human supervisor can also assist the robot in eliminating execution errors based on his/her situation awareness (SA). Through experimental validation, we show that including a human supervisor in the loop for grasping tasks in low-cost robots outperforms full autonomy. In fact, our proposed system showed a marked performance improvement by increasing the end-to-end success rate of the baseline approach, which we implemented JISA on, from 35.0% to 87.6%.