Joint-Modal Graph Convolutional Hashing for unsupervised cross-modal retrieval

Cross-modal hashing retrieval has garnered significant attention for its exceptional retrieval efficiency and low storage consumption, especially in large-scale data retrieval. However, due to the difference in modality and semantic gap, the existing methods fail to fuse multi-modal information effectively or adjust weight adaptively, which further damages the discriminative ability of the generated hash code. In this paper, we propose an innovative approach called the Joint-Modal Graph Convolutional Hashing (JMGCH) method via adaptive weight assignment for unsupervised cross-modal retrieval. JMGCH consists of a Feature Encoding Module (FEM), a Joint-Modal Graph Convolutional Module (JMGCM), an Adaptive Weight Allocation Fusion Module (AWAFM), and a Hash Code Learning Module (HCLM). After the image and text have been encoded, we use the graph convolutional network to further explore the semantic structure. To consider both the intra-modal and inter-modal semantic relationships, JMGCM is proposed to capture the correlations of different modalities, and then fuse the features from uni-modality and cross-modality by designed AWAFM. Finally, in order to obtain the hash code with greater expressive capacity, the features of one modality are used to reconstruct the features of another one, so as to reduce the gap between different modalities. We conduct extensive experiments on three widely used cross-modal retrieval datasets, and the results demonstrate that our proposed framework achieves satisfactory retrieval performance.