Robotic Slip Detection with Visual-Tactile Fusion Based on 3D Attention Mechanism

During the robot grasping and operation process, the sliding problem will occur between the grasping object and the end tool. To solve this problem, this paper proposes a visual-tactile fusion slip detection model based on the 3D attention mechanism SimAM and an improved C3D convolutional neural network (C3D-VTSimAM). First, we set different widths and strengths to lift fixed heights to grab different objects, and obtain visual and tactile time series data to build a visual-tactile fusion slip detection data set and use it to train and test the network; then, the 3D attention mechanism SimAM is introduced to extract features from the visual and tactile data, and then fuse the visual and tactile feature vectors; fuse the visual and tactile feature vectors The experimental results show that the correct rate of robot grasping slip detection results reaches 98.37%, which is 4+% higher than that of the original C3D network and the introduction of CBAM and Self-Attention, which proves the effectiveness of the method, and provides a better reference value and theoretical support for the robot in terms of stability grasping and fine operation on tasks.