Model-Free Dynamic Consensus in Multi-Agent Systems: A Q-Function Perspective

This paper presents a new method for dynamic consensus in linear discrete-time homogeneous multi-agent systems (MAS). Achieving state consensus in such systems involves constraints linked to the graph's spectral properties, complicating the design of coupling gains, especially in large-scale networks. The proposed approach reformulates the dynamic consensus problem with a prescribed convergence rate by introducing a state–action value function within a synthetic linear quadratic regulation (LQR) framework, thereby expressing the problem as a semidefinite program (SDP). The resulting SDP supports the joint design of local feedback and coupling gains in both model-based and model-free settings. To handle non-convex feasibility conditions, a convex–concave decomposition strategy is developed, guaranteeing convergence to a stationary point. In the fully model-free case, the method eliminates the need for system identification or explicit knowledge of agent dynamics, relying solely on input–state data to construct an equivalent data-driven SDP. Finally, a new algorithm balancing feasibility, convergence rate, and energy efficiency enhances design flexibility. Numerical results demonstrate the effectiveness of the proposed method in diverse scenarios.