Dynamic production scheduling towards self-organizing mass personalization: A multi-agent dueling deep reinforcement learning approach

Mass personalization is rapidly approaching. In response, manufacturing systems should be capable of autonomously changing production plans, configurations and schedules under dynamic manufacturing environments for producing personalized products. Self-organizing manufacturing network is a promising paradigm for mass personalization. The backbone of a self-organizing manufacturing network is an adaptive production scheduling method to dynamically allocate and sequence manufacturing jobs under dynamic settings, such as stochastic processing time or unplanned machine breakdown. However, existing production scheduling methods (i.e., heuristic rules, meta-heuristic algorithms, and existing reinforcement learning models) fail to automatically optimize production schedules while maintaining stable manufacturing performance, under dynamic settings. In this paper, we designed a reinforcement learning-based static-training-dynamic-execution approach for dynamic job shop scheduling problems. The scheduling policies are learned from static scheduling instances by a multi-agent dueling deep reinforcement learning approach. Under this approach, we proposed new representations of observation, action, reward, and cooperation mechanisms between agents. The learned scheduling policies are then deployed to a dynamic scheduling system where stochastic processing time and unplanned machine breakdown randomly occur. Extensive simulation experiments demonstrated that our approach outperforms heuristic rules on makespan under two dynamic manufacturing settings.