Multiagent Reinforcement Learning for Nash Equilibrium Seeking in General-Sum Markov Games

This article studies the problem of noncooperative multiagent reinforcement learning (MARL), where selfish agents play a general-sum Markov game. We consider the framework where no agent has explicit information on the model of dynamic environment, the model of other agents, and even on its own cost function. We propose an actor–critic MARL to learn the Nash equilibrium (NE) policy of the agents. The main contribution of this article is to extend the NE seeking methods to incomplete information stochastic nonzero sum games. Based on such formulation and under some conventional assumptions, we prove that by applying linear function approximators, the policies of agents converge to an approximation of the first-order NE point of the game. Finally, as a case study, the framework is applied to a Cloud Radio Access Network.