强化学习
数学优化
约束(计算机辅助设计)
最大熵原理
计算机科学
熵(时间箭头)
约束满足
趋同(经济学)
单调函数
约束优化
缩小
最优化问题
最优控制
惩罚法
人工智能
交叉熵法
约束规划
任务(项目管理)
系统安全
约束学习
交叉熵
约束优化问题
作者
Qiwei Liu,Ye Yuan,Lingyue Zhang,Kaitian Chen,Yunkai Lv,Sheng Gao,Huaicheng Yan
标识
DOI:10.1609/aaai.v40i35.40198
摘要
Deploying multi-agent reinforcement learning (MARL) in safety-critical systems faces significant challenges due to insufficient agent exploration and inadequate safety constraint guarantees. Current approaches are constrained by two fundamental limitations: inefficient exploration leading to suboptimal policies, and expected-cost-based constraint frameworks failing to ensure full-process safety. To address these challenges, this paper proposes a novel safety-aware maximum entropy MARL framework using Conditional Value-at-Risk (CVaR) as a joint safety metric, which quantifies constraint satisfaction under worst-case scenarios for multi-agent systems. Moreover, we develop the Worst-Case Multi-Agent Soft Actor-Critic (WCMASAC) algorithm, incorporating sequential update mechanisms and maximum entropy optimization for heterogeneous agents, enhanced with distributed safety critics. Theoretically, we establish the monotonic improvement property, guaranteed constraint satisfaction, and convergence to a generalized Nash equilibrium for WCMASAC. Extensive experiments on Safety-Gymnasium based benchmarks demonstrate that WCMASAC outperforms state-of-the-art baselines in both task reward acquisition and safety constraint violation reduction, while exhibiting superior exploration efficiency and risk-aware control capabilities.
科研通智能强力驱动
Strongly Powered by AbleSci AI