Mastering the Working Sequence in Human-Robot Collaborative Assembly Based on Reinforcement Learning

A long-standing goal of the Human-Robot Collaboration (HRC) in manufacturing systems is to increase the collaborative working efficiency. In line with the trend of Industry 4.0 to build up the smart manufacturing system, the collaborative robot in the HRC system deserves better designing to be more self-organized and to find the superhuman proficiency by self-learning. Inspired by the impressive machine learning algorithms developed by Google Deep Mind like Alphago Zero, in this paper, the human-robot collaborative assembly working process is formatted into a chessboard and the selection of moves in the chessboard is used to analogize the decision-making by both human and robot in the HRC assembly working process. To obtain the optimal policy of the working sequence to maximize the working efficiency, agents in the system are trained with a self-play algorithm based on reinforcement learning, without guidance or domain knowledge beyond game rules. A convolution neural network (CNN) is also trained to predict the distribution of the priority of move selections and whether a working sequence is the one resulting in the maximum of the HRC efficiency. A height-adjustable standing desk assembly is used to demonstrate the proposed HRC assembly algorithm and its efficiency in real-time task planning.