ADMM-Enhanced CNN Training Framework With Global Convergence Guarantees

Convolutional Neural Networks (CNNs) are widely employed across various domains due to their exceptional capability in extracting complex features from data. Traditional CNN training predominantly relies on gradient-based algorithms. However, these methods face several limitations, including issues such as vanishing and exploding gradients, which hinder efficient training, particularly in deeper networks. The Alternating Direction Method of Multipliers (ADMM) offers a promising alternative, having demonstrated considerable success in conventional machine learning applications. As a gradient-free algorithm, ADMM effectively circumvents the challenges associated with gradient-related problems, positioning it as a potential substitute for gradient-based methods. Nevertheless, the inherent differences between convolution operations and matrix multiplications present significant challenges for directly applying ADMM to CNN training. To address these challenges, we propose a novel framework, termed ADMM-CNN, that leverages the ADMM algorithm for CNN training. Our approach employs the im2col technique to transform convolution operations into matrix multiplications, thereby streamlining the optimization process. Within the subproblems, we utilize local linear approximation to facilitate effective parameter updates. Additionally, we provide a rigorous theoretical convergence analysis to validate the stability and reliability of our proposed method. We evaluate the performance of ADMM-CNN through comprehensive experiments on two benchmark datasets and a real-world dataset. The experimental results demonstrate that our framework not only achieves competitive performance but also requires fewer iterations compared to conventional gradient-based methods, highlighting the efficiency and practicality of our approach.