Augmenting Vision-Based Grasp Plans for Soft Robotic Grippers using Reinforcement Learning

Vision-based techniques for grasp planning of robotic end-effectors have been successfully deployed in pick-and-place tasks. However, for computing the optimal grasp, they assume the gripper to be of a rigid parallel-jaw type or a single-point vacuum suction-based design. Planners for soft robotic grippers have used learning from demonstration or heuristics that rely on the compliance of the gripper to achieve a grasp. We demonstrate a model-free reinforcement learning (RL) approach that modifies vision-based grasp plans generated for parallel-jaw grippers and adapts them to grasping with a four-fingered soft gripper. The observed state of the RL model is comprised of the grasp plans from the vision module, the deformation of the fingers, and the pose of the end-effector. The RL model controls each finger separately, discovering grasp synergies during training. This approach is compared to a baseline grasp synergy where all four fingers simultaneously enclose the object. In simulation, we achieve a pick-and-place success rate of 58.4% with the RL model for top-down grasping, compared to 43.2% with the baseline grasp synergy.