A Multiple Triplet-Ranking Model for Fine-Grained Sketch-Based Image Retrieval

Fine-grained sketch-based image retrieval (FG-SBIR) addresses the problem of matching an input sketch with a specific photo containing the same instance. The key challenge of learning a FG-SBIR model is to bridge the domain gap between photo and sketch. Most existing approaches build a joint embedding space where two domains can be directly compared. They only focus on the highly abstract features in final fully connected (FC) layer, ignore some low-level semantic concepts in convolutional layers. In this paper, we propose a multiple triplet-ranking model in FG-SBIR task. Specially, we introduce an auxiliary supervision loss function in the convolutional layer, and we use the fusion of features from convolutional layer and final FC layer to build the joint embedding space. Extensive experiments show that the proposed multiple triplet-ranking model significantly outperforms the state-of-the-art.