Reinforcement learning-based dynamic production-logistics-integrated tasks allocation in smart factories

In Industry 4.0, the production planning and execution of smart factories (SFs) full of continuously delivered small-batch orders become dynamic and complicated. Traditional centralised manufacture planning is difficult to handle unexpected disturbances. With the aid of new information technologies, resources in SFs become smart and connected to make autonomous decisions. This paper tries to release intelligence of smart connected resources to allocate production tasks and logistics tasks in SFs coordinately and autonomously. The architecture is modelled as an autonomous decision-making manufacturing system with IIoT support, which aims to synchronously allocate manufacturing tasks by the bidding of resources in SFs. Then, a dynamic production-logistics-integrated tasks allocation model is built. The orders makespan and resources utilisation are considered as the objective function, and production resources and logistics resources are integrated to autonomously communicate and interact with each other to bid for dynamic production-logistics integrated operations. To figure out, a reinforcement learning (RL) algorithm is studied, which makes operations decisions for each job step by step based on in-situ data during manufacturing process. Finally, a demonstrative case showed that compared to centralised scheduling system, the RL-based model performs better in handling production-logistics-integrated tasks allocation problem in SFs full of dynamic and small-batch individualised orders.