Robot Grasp with Multi-object Detection based on RGB-D Image

An autonomous grasping system based on the method of depth learning is built in this paper, which is composed of a depth camera, a two-finger parallel gripper and a six-degree-of-freedom mechanical arm. To solve the poor performances of recognition and location of the grasping system in complex scenes with multiple irregular objects of different size, a multi-object grasping detection method based on RGB-D images is proposed. This method uses the combination of FPN and RPN methods to design the grasp proposal network, which can simultaneously use the shallow and deep information in the network to improve the detection effect of objects with different sizes, especially small objects. Secondly, the ROI Align method is introduced as a regional feature aggregation method, and the bilinear interpolation method can significantly improve the detection accuracy of the grasping model. Finally, the multi-object dataset containing small objects is used to test the proposed model, and the result illustrates the generalization ability of the proposed model. Experimental results show that the accuracy of the method proposed in this paper achieves 97% accuracy on single-object recognition and 92% accuracy on multi-object recognition. The robot grasping system based on Kinect camera and UR5 robotic arm achieved 91.0% grasping success rate on single-object, and 89% grasping success rate on multi-object. Comparing with some popular grasping methods, the results show the effectiveness and superiority of the proposed method in this paper.