Attention-based Cropping and Erasing Learning with Coarse-to-Fine Refinement for Fine-grained Visual Classification

• Attention regions cropping and erasing data augmentation approaches are proposed for fine-grained visual classification. • A coarse-to-fine refinement strategy is proposed to refine the classification result with the defined confidence value. • Analyses of three challenging fine-grained datasets along with currently outstanding methods. • The comprehensive experimental results on three challenging FGVC datasets show the effectiveness of our approach. Fine-grained visual classification is challenging due to similarities within classes and discriminative features located in subtle regions. Conventional methods focus on extracting features from the most discriminative parts, which may underperform when these parts are occluded or invisible. And the limited training data also leads to serious overfitting problem. In this paper, we propose an Attention-based Cropping and Erasing Network (ACEN) with a coarse-to-fine refinement strategy to address these problems. By convolving the feature maps from CNN, we obtain a set of attention maps which focus on discriminative object parts. Guided by the attention maps, we propose attention region cropping and erasing operations to augment training data. Moreover, the attention region cropping enhances local discriminative feature learning, and the attention region erasing promotes multi-attention learning. During inference phase, the coarse-to-fine refinement strategy is proposed to refine the model prediction. Extensive experiments demonstrate that our approach achieves state-of-the-art performance on challenging benchmarks, including CUB-200-2011, FGVC-Aircraft and Stanford Cars.