Multi-agent reinforcement learning with graph convolutional neural networks for optimal bidding strategies of generation units in electricity markets

Finding optimal bidding strategies for generation units in electricity markets would result in higher profit. However, it is a challenging problem due to the system uncertainty which is due to the lack of knowledge of the strategies of other generation units. Distributed optimization, where each entity or agent decides on its bid individually, has become state of the art. However, it cannot overcome the challenges of system uncertainties. Deep reinforcement learning is a promising approach to learning the optimal strategy in uncertain environments. Nevertheless, it is not able to integrate the information on the spatial system topology into the learning process. This paper proposes a semi-distributed learning algorithm based on deep reinforcement learning (DRL) combined with a graph convolutional neural network (GCN). In fact, the proposed framework helps the generation units to update their decisions by getting feedback from the environment so that they can overcome the challenges of uncertainties. In this proposed algorithm, the state and connection between nodes are the inputs of the GCN, which can make generation units aware of the network structure of the system. This information on the system topology helps the generation units learn to improve their bidding strategies and increase their profit. We evaluate the proposed algorithm on the IEEE 30-bus system under different scenarios. Also, to investigate the generalization ability of the proposed approach, we test the trained model on the IEEE 39-bus system. The results show that the proposed algorithm has a better generalization ability compared to the DRL and can result in a higher profit when changing the topology of the system.