Dynamic label correlations and dual-semantic enhancement learning for cross-modal retrieval

With the rapid growth of multi-modal data, Cross-Modal Hashing (CMH) is widely applied due to its outstanding performance in both search and storage. Nevertheless, there are two problems required to be addressed, namely, (1) most existing methods neglect to dynamically learn the importance of different labels; (2) many methods fail to purify the consistency of data extracted from different feature spaces. To tackle these issues, we propose Dynamic Label Correlations and Dual-Semantic Enhancement Learning for Cross-Modal Retrieval (DLCDE) in this study. DLCDE mainly consists of two parts: Label Semantic Enhancement with Dynamic Label Reconstruction (LSEDLR) and Sample Semantic Enhancement with Consistency Purification and Structure Maintenance (SECPSM). LSEDLR first utilizes label-wise self-expression to dynamically explore the latent correlations between different labels and then employs a graph-based manifold regularizer to explore the structural relationships in the transformed label space to enhance label semantics. SECPSM leverages Hadamard-Product-based Matrix Factorization to enhance the common relationships between samples, thereby enhancing the sample semantics of the latent shared space. Ultimately, dual-semantic enhancement learning is achieved by integrating enhanced label semantics and sample semantics in Distance-Distance Difference Minimization (DDDM). Numerous experiments on four benchmark datasets reveal that DLCDE surpasses a number of state-of-the-art CMH methods. The source code for DLCDE is publicly available at https://github.com/Fizzyf/DLCDE .