A Sensory Soft Robotic Gripper Capable of Learning-Based Object Recognition and Force-Controlled Grasping

Soft robotic grippers possess high structural compliance and adaptability, allowing them to grasp objects with unknown and irregular shapes and sizes. To enable more dexterous manipulation, soft sensors that are similar in mechanical properties to common elastomer materials are desired to be integrated into soft grippers. In this paper, we develop ionic hydrogel-based strain and tactile sensors and integrate these sensors into a three-finger soft gripper for learning-based object recognition and force-controlled grasping. Such hydrogel-based sensors have excellent conductivity, high stretchability and toughness, good ambient stability, and unique antifreezing property; they can be readily attached to a soft gripper at desired locations for strain and tactile sensing. By using a deep-learning model, the sensory soft gripper is demonstrated to be capable of grasping and recognizing objects at both room and freezing temperatures, and achieving close to 100% recognition accuracy for ten typical objects. Moreover, the capacitive tactile feedback of the gripper is utilized to develop a closed-loop force controller and realize force-controlled grasping of fragile or highly deformable objects. A new slip detection and compensation strategy is also proposed and validated for the sensory gripper for adjusting the grasping force in real time upon detecting slippage. Note to Practitioners—The multimodal sensation of a soft robotic gripper could enrich its grasping functionalities and improve its manipulation performance. This research integrates novel antifreezing ionic hydrogel-based strain and tactile sensors into a three-finger soft robotic gripper for learning-based object recognition and force-controlled grasping. Constructed from a highly stretchable, ambient-stable, and antifreezing ionic hydrogel, the strain and tactile sensors can be readily integrated at the desired locations on the soft gripper, and can reliably operate at both ambient and freezing temperatures with excellent mechanical and electrical properties. Based on the feedback of the strain and tactile sensors, a deep learning model is employed to enable high-accuracy object recognition while grasping, which can be useful for manipulation in vision-free environments. Closed-loop force control and slip compensation strategies are also demonstrated for reliably grasping fragile/deformable objects and handling slip events during the manipulation of heavy objects. The sensory soft gripper and the associated object recognition and force control methods could find practical applications in a variety of robotic manipulation tasks.