Re-parameterized Global Channel Interaction Attention Module for Convolutional Neural Networks

In this paper, a compact and effective attention module for convolutional neural networks (CNNs), called re-parameterized global channel interaction attention module (GCIAM), is presented to achieve better communication of channels. Firstly, we propose the global channel receptive filed (GCRF) to measure the number of adjacent channels participating in inter-channel interactions. Secondly, we adopt multi-scale 1-D convolutions to achieve better coverage of the GCRF. Specifically, our method aggregates the information along channel and then encodes cross-channel relationships via multi-scale 1-D convolutions consist of large kernels and small kernels, which enables easier access to different scales of interactions from a large GCRF and significantly reduces the model complexity. Finally, we develop a structural re-parameterization technique for multi-scale 1-D convolutions to boost the inference speed. Extensive experiments on standard benchmarks verify the superiority of our proposed method over various state-of-the-art (SOTA) counterparts. Our method stably outperforms other attention mechanisms by a large margin at almost no costs.