一般化
极小极大
理论(学习稳定性)
计算机科学
随机梯度下降算法
数学优化
超参数
人口
趋同(经济学)
梯度下降
人工智能
最优化问题
机器学习
极大极小近似算法
近似算法
随机逼近
数学
随机优化
数据建模
算法
泛化误差
作者
Miaoxi Zhu,Yan Sun,Li Shen,Bo Du,Dacheng Tao
标识
DOI:10.1109/tpami.2026.3677027
摘要
Minimax optimization is gaining increasing attention in modern machine learning applications. Driven by large-scale models and massive volumes of data collected from edge devices, as well as the concern to preserve client privacy, distributed minimax optimization algorithms become popular, such as Local Stochastic Gradient Descent Ascent (Local-SGDA), and Local Decentralized SGDA (Local-DSGDA). While most existing research on distributed minimax algorithms focuses on convergence rates and communication efficiency, their generalization performance remains largely unexplored, whereas generalization ability is a pivotal indicator for evaluating the holistic performance of a model when fed with unknown data. In this paper, we propose the stability-based generalization analytical framework for Distributed-SGDA, which unifies two popular distributed minimax algorithms including Local-SGDA and Local-DSGDA, and conduct a comprehensive analysis of stability error, generalization gap, and population risk across different metrics under various settings, e.g., (S)C-(S)C, PL-SC, and NC-NC cases. Our theoretical results reveal the trade-off between the generalization gap and optimization error and suggest hyperparameters choice to obtain the optimal population risk. Numerical experiments for Local-SGDA and Local-DSGDA validate the theoretical results.
科研通智能强力驱动
Strongly Powered by AbleSci AI