Multi-Agent Distributed Reinforcement Learning Algorithm for Free-Model Economic-Environmental Power and CHP Dispatch Problems

In conventional methods for Economic Dispatch Problem (EDP) and Economic-Environmental Dispatch Problem (EEDP), a full connection between the units and the control center is considered, while some faults in the communication system are possible in practical conditions. Hence, an intelligent and high-speed distributed method that is robust against disconnection is needed. In this paper, a Multi-Agent Distributed Reinforcement Learning (MADRL) algorithm based on consensus control for EDP and EEDP is presented. In this method, the incremental cost of units is optimized based on Lagrange method and the reinforcement learning algorithm. Thus, a performance index is defined for each agent in proposed algorithm to be independent of the unit model. The performance index of each agent is the sum of two terms, including i) the difference between the previous performance index value and local power and heat mismatch values and ii) the sum of the difference between the incremental cost of the unit with other neighboring units. In other words, optimizing the defined performance indexes of the agents eliminates the need for the system model. The MADRL method is tested on several grids and compared with other methods. The numerical results show an improvement in the algorithm speed and optimal point.