A heterogeneous graph attention-enhanced deep reinforcement learning framework for flexible job shop scheduling problem with variable sublots

Variable lot-sizing is an effective approach to improve production efficiency by splitting an operation into several sublots, which has been widely applied in flexible manufacturing systems. However, the flexibility of lot-sizing will dramatically expand the solution space, leading to excessive computation time in converging to the relative optimum. To address this challenge, this paper introduces an end-to-end deep reinforcement learning framework based on heterogeneous graph attention mechanisms (HGADRL) for flexible job shop scheduling problem with variable sublots. Unlike traditional heuristic and rule-based methods, HGADRL dynamically learns the high-dimensional nature, providing a more generalizable solution in a very short time. In HGADRL, a modified heterogeneous disjunctive graph is designed to represent the dynamic scheduling status, including operation selection and sublot division. A dual-scale graph attention network combined with two interconnected attention modules is developed, enabling the precise capture of complex interdependencies between heterogeneous vertices. This approach can significantly enhance the agent's ability to self-learn and evolve optimal policies. By leveraging local and global features extracted through the graph attention network, an actor-critic network is employed for high-quality scheduling in different states. Experimental results demonstrate that the proposed method outperforms the 12 mixed priority dispatching rules, two meta-heuristic methods and two deep reinforcement learning methods in all 500 synthetic instances. Additionally, the proposed method outperforms all compared methods across 16 unseen scales of instances and four real-world instances, demonstrating its strong generalization capabilities. • The flexible job shop scheduling problem with variable sublots is studied via DRL. • A heterogeneous disjunctive graph for representing solutions of FJSP-VS. • A customized MDP for FJSP-VS to accurately represent states and composite actions. • The heterogeneous dual-scale graph attention network for deep features extraction.