亲爱的研友该休息了!由于当前在线用户较少,发布求助请尽量完整地填写文献信息,科研通机器人24小时在线,伴您度过漫漫科研夜!身体可是革命的本钱,早点休息,好梦!

A maximum entropy deep reinforcement learning method for sequential well placement optimization using multi-discrete action spaces

强化学习 最大熵原理 钢筋 计算机科学 人工智能 熵(时间箭头) 数学优化 数学 工程类 结构工程 物理 量子力学
作者
Kai Zhang,Zifeng Sun,Liming Zhang,Guojing Xin,Zhongzheng Wang,Wenjuan Zhang,Piyang Liu,Xia Yan,Huaqing Zhang,Yongfei Yang,Hai Xia Sun
出处
期刊: [Elsevier BV]
卷期号:240: 213004-213004 被引量:15
标识
DOI:10.1016/j.geoen.2024.213004
摘要

Well placement optimization is a crucial method to solve the planar conflicts in reservoir development, mainly to determine the optimal well locations and drilling sequence to maximize the economic benefits of reservoir development. However, the current well placement optimization methods face the problems of high-dimensional discretization of optimization variables and lack of effective incentives for policy exploration, which make it challenging to improve the global optimization ability (the ability to jump out of locally optimal solutions in time and continuously search for better solutions in the whole optimization process) and real-time adjustability of the well placement optimization methods under limited numerical simulation times. In this paper, we propose a new sequential well placement optimization method, based on the Discrete Soft Actor-Critic Algorithm (DSAC), which incorporates the maximum entropy mechanism to formulate well placement and drilling sequencing schemes more efficiently and maximize the net present value (NPV) over the entire life cycle of reservoir development. Specifically, the method models the well placement optimization problem as a Markov Decision Process (MDP) and achieves sequential well placement optimization by training a Deep Reinforcement Learning (DRL) agent that maps reservoir states to a stochastic policy of well placement variables as well as evaluates the value function of the current policy. The DRL agent can determine the optimal infill well location in real-time based on the reservoir state at different times during the development process, thus obtaining the optimal drilling sequence. The proposed method in this paper has two innovations. First, by reconstructing the large-scale discrete action space of well placement optimization variables into multi-discrete action spaces, and with the maximum entropy mechanism, policy exploration is encouraged to improve the global optimization capability. Second, the trained policy can swiftly adapt the subsequent well placement scheme for a specific state of the target reservoir without the requirement to initiate training from scratch, which can realize the offline application of the trained policy and has better real-time adjustability. To verify the effectiveness of the proposed method, it is tested in 2D and 3D reservoir models. The results show that DSAC not only outperforms the gradient-based optimization method, classical evolutionary algorithms, and existing reinforcement learning proximal policy optimization (PPO) method in terms of global optimization ability but also shows better real-time adjustability of the trained policy when applied offline.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
舒适的夜柳完成签到,获得积分10
1秒前
4秒前
芦苇红柳完成签到 ,获得积分10
14秒前
Shicheng完成签到,获得积分10
48秒前
平常觅风完成签到,获得积分10
56秒前
自由山槐完成签到,获得积分10
1分钟前
GingerF的应助被ZZWZHENG采纳,获得100
1分钟前
眯眯眼的破茧完成签到,获得积分10
1分钟前
1分钟前
whistle发布了新的文献求助10
1分钟前
酷炫梦山完成签到,获得积分10
1分钟前
鲤鱼遥完成签到,获得积分10
1分钟前
1分钟前
whistle发布了新的文献求助10
2分钟前
开朗的翠霜完成签到,获得积分10
2分钟前
欣喜寒烟完成签到,获得积分10
2分钟前
欣喜寒烟关注了科研通微信公众号
2分钟前
fantasy的应助被科研通管家采纳,获得10
2分钟前
落寞的冰海完成签到,获得积分10
2分钟前
呆萌的若剑完成签到,获得积分10
2分钟前
2分钟前
内向鸣凤完成签到,获得积分10
3分钟前
xxz完成签到 ,获得积分10
3分钟前
zhangxiaoqing完成签到,获得积分10
3分钟前
调皮的手套完成签到,获得积分10
3分钟前
沉静友蕊完成签到,获得积分10
3分钟前
乐乐的应助被ZZWZHENG采纳,获得50
4分钟前
自由完成签到,获得积分10
4分钟前
parantal发布了新的文献求助10
4分钟前
喜悦如萱完成签到,获得积分10
4分钟前
英俊的傲珊完成签到,获得积分10
4分钟前
4分钟前
粗暴的莹发布了新的文献求助10
4分钟前
4分钟前
wangch198201发布了新的文献求助10
4分钟前
焚天尘殇完成签到,获得积分10
4分钟前
5分钟前
ZZWZHENG发布了新的文献求助50
5分钟前
5分钟前
5分钟前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
John McDowells Bildungstheorie 800
八旗制度の研究 600
plantTFDB 508
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8173069
求助须知:如何正确求助?哪些是违规求助? 9565519
关于积分的说明 21780424
捐赠科研通 7765088
什么是DOI,文献DOI怎么找? 3431536
关于科研通互助平台的介绍 2555638
邀请新用户注册赠送积分活动 2476668