Solving job scheduling problems in a resource preemption environment with multi-agent reinforcement learning

In smart manufacturing, robots gradually replace traditional machines as new processing units, which have significantly liberated laborers and reduced manufacturing expenditure. However, manufacturing resources are usually limited so that the preemption relationship exists among robots. Under this circumstance, job scheduling puts forward higher requirements on accuracy and generalization. To this end, this paper proposes a scheduling algorithm to solve job scheduling problems in a resource preemption environment with multi-agent reinforcement learning. The resource preemption environment is modeled as a decentralized partially observable Markov decision process, where each job is regarded as an intelligent agent that chooses an available robot according to its current partial observation. Based on this modeling, a multi-agent scheduling architecture is constructed to handle the high-dimension action space issue caused by multi-task simultaneous scheduling. Besides, multi-agent reinforcement learning is employed to learn both the decision-making policy of each agent and the cooperation between job agents. This paper is novel in addressing the scheduling problem in a resource preemption environment and solving the job shop scheduling problem with multi-agent reinforcement learning. The experiments of the case study indicate that our proposed method outperforms the traditional rule-based methods and the distributed-agent reinforcement learning method in total makespan, training stability, and model generalization.