A Hierarchical Multi-Action Deep Reinforcement Learning Method for Dynamic Distributed Job-Shop Scheduling Problem With Job Arrivals

计算机科学 强化学习 调度(生产过程) 马尔可夫决策过程 作业车间调度 工作车间 分布式计算 动态优先级调度 汽车工业 流水车间调度 工业工程 人工智能 运筹学 马尔可夫过程 数学优化 工程类 计算机网络 数学 地铁列车时刻表 统计 服务质量 操作系统 航空航天工程
作者
Jiang‐Ping Huang,Liang Gao,Xinyu Li
出处
期刊:IEEE Transactions on Automation Science and Engineering [Institute of Electrical and Electronics Engineers]
卷期号:22: 2501-2513 被引量:50
标识
DOI:10.1109/tase.2024.3380644
摘要

The Distributed Job-shop Scheduling Problem (DJSP) is a significant issue in both academic and industrial fields. In real-world production, uncertain disturbances such as job arrivals are inevitable. In the paper, the DJSP with job arrivals is addressed with a Multi-action Deep Reinforcement Learning (MDRL) method. Firstly, a multi-action Markov Decision Process (MDP) is formulated, where a hierarchical multi-action space combining operation set and factory set is proposed. The reward function is related to the machine idle time. Additionally, the state transition is also elaborately designed, which includes four typical cases based on job arrival times. Then, a scheduling policy with two decision networks is proposed, where the Graph Neural Network (GNN) is applied to extract the intrinsic information of the scheduling scheme. A Proximal Policy Optimization (PPO) with two actor-critic frameworks is designed to train the model to achieve intelligent decision-making with hierarchical action selections. Extensive experiments are conducted based on 1350 instances. The comparison among 17 composite rules, 3 closely-rated DRL methods, and 2 metaheuristics has proven the outperformance of the proposed MDRL. The application of the MDRL in an automotive engine manufacturing company has demonstrated its engineering value in the industrial field. Note to Practitioners —The DJSP with job arrivals is a common challenge faced by equipment manufacturers, specifically in the electronic device manufacturing industry. These manufacturers are located in different areas and have varying facility configurations and operation trajectories. To address this challenge, a machine learning-based method can be applied for scheduling daily production tasks. This method divides the DJSP into two subproblems, namely job assigning and job sequencing, and uses two decision networks based on DRL to solve them. To address the uncertainty caused by job arrivals, the rescheduling process and the state update mechanism are carefully designed. A GNN is used for feature extraction at each decision point, and it feeds the decision networks with the extracted features to make the optimal selection. The proposed method has the ability of self-learning and self-adapting, and its effectiveness has been proven through experiments on 1350 test instances. Its practical application has been demonstrated in the production scenarios of an automotive engine manufacturing company. In the future, the method can be adopted to solve more complex distributed manufacturing problems that have constraints such as transportation costs and machine breakdowns.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
2032jia完成签到,获得积分10
刚刚
李健的小迷弟的应助被李华采纳,获得10
刚刚
刚刚
刚刚
称心的灭龙完成签到,获得积分10
1秒前
喵喵完成签到 ,获得积分10
1秒前
zhangzhima完成签到,获得积分10
1秒前
卫夫人发布了新的文献求助10
1秒前
null完成签到,获得积分10
1秒前
小黑完成签到,获得积分10
2秒前
DW的应助被YPHCC采纳,获得10
3秒前
3秒前
杨总发布了新的文献求助10
3秒前
名称不是重点完成签到,获得积分10
3秒前
如愿完成签到,获得积分10
3秒前
123完成签到 ,获得积分10
3秒前
杨艺完成签到,获得积分10
4秒前
漂亮恶天完成签到 ,获得积分10
4秒前
北天辰完成签到,获得积分10
4秒前
我是老大的应助被111采纳,获得10
4秒前
Who发布了新的文献求助30
5秒前
淡然雁易完成签到 ,获得积分10
5秒前
小贺完成签到,获得积分10
6秒前
扶摇发布了新的文献求助10
6秒前
6秒前
瓜瓜完成签到,获得积分10
6秒前
坦率白竹完成签到,获得积分10
7秒前
星星完成签到,获得积分10
7秒前
7秒前
dyy完成签到,获得积分10
8秒前
潇洒夏天完成签到,获得积分10
8秒前
李wf完成签到,获得积分10
8秒前
L1ghtshow完成签到,获得积分10
8秒前
yippee完成签到 ,获得积分10
8秒前
9秒前
9秒前
明明完成签到,获得积分10
9秒前
甜甜新竹完成签到,获得积分10
9秒前
无花果的应助被科研的熊猫采纳,获得10
9秒前
调皮平安完成签到,获得积分10
10秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
John McDowells Bildungstheorie 800
Social Psychology in the Real World 600
八旗制度の研究 600
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8180275
求助须知:如何正确求助?哪些是违规求助? 9570366
关于积分的说明 21810725
捐赠科研通 7776380
什么是DOI,文献DOI怎么找? 3433277
关于科研通互助平台的介绍 2557483
邀请新用户注册赠送积分活动 2479042