FVSight: A Novel Multimodal Tactile Sensor for Robotic Object Perception

Haptic perception is an important capability for robots to perform tasks in unknown environments, but existing haptic sensors focus on the fingertips of the robot, making the sensing area of the sensor limited, and only one haptic modality can be collected with one sensory layer. In this paper, we propose a multimodal tactile sensor, the FVSight, that is based on a distributed flexible tactile sensing array for daily objects. The sensor is fixed at the end of the robotic arm and touches the object under test by pressing, which can collect both tactile image and force-tactile array information of the object using one perceptual layer. Meanwhile, spatially, the visual contact area of the sensor when pressing the object is the same as the tactile contact area, and temporally, the two heterogeneous tactile modal information is collected simultaneously, which effectively solves the problem of weak pairing of visual-tactile information in conventional tactile sensors. We also use extensive experiments to demonstrate the great potential of the FVSight sensor in the direction of robotic object perception.