End-effector camera usage investigation for dual-arm industrial robot

For robot perception system, accuracy and cost coexist. For single camera perception system, the cost will be very high to achieve high perception accuracy. Thus, the cooperation of several cameras can be a possible solution to balance the cost and accuracy. \nThis report aims to investigate the use of end-effector camera to increase the accuracy of object recognition for dual-arm industrial robot, NEXTAGE. It covers the key parameters for robot perception system, like camera calibration, stereo camera usage, hand camera usage, and robot motion control. The robot integrates with a Kinect stereo camera, two hand cameras, and two grippers. The cameras provide 3D depth and 2D image data, which provide the vision system for the robot. The grippers allow user to implement the desired application. \nThe software development for the robot was done in Robot Operation System (ROS) which is embedded in Linux operation system, Ubuntu 14.04. In this project, the author has provided the necessary knowledge and background for the functions development of the robot, like OpenCV, Point Cloud Library. In addition, the author provided the way of implementing the C++ or Python code into ROS, which enables the robot to handle real time control tasks. \nIn the project, the author has tested the method of coordination of head stereo camera and hand RGB camera in the sensing accuracy improvement. All the codes have been integrated into ROS structure, and each node was intended to perform one or more basic functions. The robot was able to execute the task successfully and achieve desired outcomes with the code developed.