Siamese Convolutional Neural Network Based Visual Servo for Manipulator

A visual servo algorithm based on Siamese Convolution Neural Network is proposed for the manipulator to avoid the requirement of feature extraction and feature matching in the traditional image-based visual servo (IBVS). The algorithm feeds the current image and the desired image into the network at the same time, and outputs the relative pose difference between the two images. A closed-loop control system is constructed through the pose difference, and control the end-effector of the manipulator to reach the desired position to grasp the target workpiece. Meanwhile, in order to meet the large amount of data needed in training the neural network, an algorithm to automatically generate the data set is proposed, which can avoid manual collection and labeling of the data set and greatly save the cost. The simulations show the effectiveness and accuracy of the proposed method by comparing with the traditional feature point based IBVS, and the grasping experiment shows the feasibility of the proposed method in actual practice.