A Multi-Agent Self-Supervised State Representation Framework for Automated Algorithm Configuration

Automated algorithm configuration is an emerging area in evolutionary algorithms (EAs), where reinforcement learning (RL) has been increasingly adopted. However, most existing RL-assisted EAs still rely on simple handcrafted statistical information to construct states, which is often problem-dependent and may fail to capture complex search dynamics. To address this issue, this paper proposes a multi-agent self-supervised state representation framework with auxiliary guidance for providing agent-specific states to policy networks in RL-assisted EAs. Specifically, population observations collected online during the early stage of evolution are used to train a state encoder equipped with a multi-head readout. The encoder learns shared population tokens from these observations, while the multi-head readout derives agent-specific states, and both components are trained by integrating contrastive self-supervision with auxiliary guidance. Once trained, the encoder and readout are deployed to generate states from current population observations for downstream policy learning. The proposed framework is instantiated in four representative RL-assisted EA testbeds and evaluated across different optimization scenarios. Comparisons with three state representation methods show that MASSR generally provides more effective states for policy learning, while ablation and transfer studies further demonstrate the effectiveness and flexibility of the proposed framework.