One to Rule Them All: Reinforcement Learning Policy Selection With Closed-Loop Sensitivity

In reinforcement learning-based control, reliable zero-shot transfer from simulation to real hardware remains a major open challenge. The offline training phase typically produces multiple models, each corresponding to a different policy, and choosing which model to deploy on physical hardware is a critical decision. However, there is no established methodology for this selection process. A common practice is to deploy the policy that achieves the highest reward in simulation, without evaluating its robustness to uncertainties. As a result, due to the sim-to-real gap, the selected policy may perform poorly when transferred to a physical robotic system, potentially leading to unsafe or undesirable behavior. This work proposes to exploit the notion of “closed-loop sensitivity” for applying it, for the first time, to the analysis and comparison of different reinforcement learning models, i.e., different policies, and to estimate their expected performance on real hardware. The proposed metric captures how the closed-loop system reacts to modeling errors, providing a quantitative measure of policy robustness. Both simulation results and real world experimental campaigns demonstrate that the proposed approach enables reliable zero-shot transfer to a real robot, while effectively handling uncertainties and reducing the risk of unsafe behaviors.