Saliency aware image cropping with latent region pair

Abstract As one of the fundamental problems for image enhancing editing, image cropping seeks the best crop with high aesthetic quality and removes extraneous areas. Many recent deep learning methods have been proposed to address the problem, but they do not reveal the intrinsic mechanism of image cropping. In this paper, we explore the latent region pair and then fulfill its potential in our proposed deep learning methodology (SAIC-Net) with saliency map for automatic image cropping. For each image, a lightweight multi-scale feature extraction network is first adopted to produce deep and informative features. Then, the features of latent region pair (ROI and ROD) are aligned and refined by the proposed saliency-aware align operators and context channel attentions. Finally, hybrid loss composed by ranking loss and Huber loss is minimized when training our model. In our experiments, to reduce the searching space for candidate crops, we conduct a saliency-aware grid cropping candidates generation method to eliminate irrelevant crops. Afterwards, we provide a thorough ablation study to better figure out the superiority of each part in our method, and conduct user study against the state-of-the-art methods on a fraction of performance metrics. The quantitative and qualitative results on three benchmark datasets demonstrate the superiority of our SAIC-Net in the task of automatic cropping.