Object Grasping Motion Generation by Attention Prediction Based on Language Instructions

In order to realize cooperative work between humans and robots, we developed a technology that understands human instructions and generates appropriate motions based on learning experience. In this paper, we proposed a motion generation method that uses an attention mechanism to extract object location information from visual information and an association mechanism to predict the indicated object. In a situation where multiple objects are placed, we confirmed that the robot accurately grasps the object indicated by a person.