Fine-grained Visual Classification by Progressive Training via Jigsaw Puzzle Permutation Learning

Fine-grained visual classification remains a challenging task due to small inter-class differences and large intraclass differences. Existing methods consider only local details of information or only global information at a certain stage. However, there methods do not consider the complementary relationship between local information of different granularity and global information of the overall object on the feature maps of different stages of the network. To solve the above problem, we propose a Progressively Trained Jigsaw Puzzle Permutation Learning network (PPL-Net). To obtain the local details of the object, we adopt a progressive training strategy to obtain the semantic information of the object. At different steps, we use different granularity of shuffled versions of the images as input. Then, the classification prediction is performed by fusing the feature representations of different granularities. We obtain the global information of the object by learning jigsaw puzzle visual permutation. Specifically, by introducing the Jigsaw Puzzle Solver module we explore the location information of the different granularity image patches. Recovery of the shuffled version of the image to the original image. Thus, at each step of the network, we supervise the network by complementary relationships between local information of different granularities and global information of the object. Experiments on three standard fine-grained classification datasets (CUB-200-2011, Stanford Cars, FGCV-Aircraft) show that the performance of our method exceeds that of most methods. The code will be available at https://github.com/Zhao-fan/PPL-Net.