Deformability-Aware Stiffness-Guided Robotic Grasping Strategy based on Tactile Sensing

The absence of contact perception in conventional robotic grasping methods used in manufacturing poses a significant challenge in handling deformable objects, elevating the risk of causing permanent object damage. This limitation restricts the application of robotic manipulation in scenarios involving both rigid and deformable objects. We thus propose a novel deformation-aware stiffness-guided robotic grasping strategy that integrates tactile and visual information to facilitate the handling of both rigid and deformable objects while minimizing deformation. Object deformability is initially classified into three categories—rigid, slightly deformable, and highly deformable—using DeformNet, a novel network that captures spatial-temporal features from the high-density tactile sensor data during pre-grasping. Subsequently, for deformable objects, stiffness distribution is inferred from tactile data during visual-guided exploration, combined with RGBD images to generate deformation-minimizing grasp poses, while rigid objects are manipulated based solely on images. To validate the DeformNet, a deformability dataset comprising 13,020 temporally segmented tactile sequences was collected during the grasping of 24 objects with various grasp poses, achieving an accuracy of 94.88%. The effectiveness of stiffness distribution inference was confirmed by comparing the inferred results of nine objects in the dataset to their true stiffness distributions, and quantifying deformation of areas with varied stiffness through simulation. Furthermore, the proposed grasping strategy was compared against visual-only grasping for cups in the real world. Results demonstrated that the stiffness-guided strategy yielded significantly different grasp poses, reducing deformations in deformable objects for stable grasping compared to visual-only approaches. As a pioneering study in grasping of both rigid and deformable objects, our work advances the contact perception capability of robots, enabling robotic manipulation and human-robot collaboration in unstructured scenarios such as domestic and service industries.