Dual-Semantic Enhancement Cross-Modal Hashing with Noisy Labels

Due to its computational efficiency and low storage requirement, cross-modal hashing (CMH) gains a lot of attention. However, there are still three issues that affect its performance: 1) most existing methods focus on learning the shared semantics between different modalities with the modality-specific semantics ignored; 2) noisy labels may further exacerbate the semantic differences of different modalities during learning; and 3) most existing methods overlook the complementarity between modality-specific labels and semantic features. To address these issues, this work develops a novel CMH method called Dual-Semantic Enhancement Cross-Modal Hashing with Noisy Labels (DSENL). DSENL consists of three parts: (a) Modality-Specific Label Recovery (MSLR) that obtains modality-specific clean labels by applying matrix decomposition with low-rank and sparse constraints to the observed labels; (b) Semantic Preservation under Label Guidance (SPLG) that enhances the quality of recovered labels by using an $l_{2,1}$ norm and maintains semantic consistency across modalities by reducing discrepancies among modality-specific labels; and (c) Dual-Semantic Enhancement Learning (DSEL) that integrates both label and sample semantics from modality-specific to enhance the discriminative capability of hash codes. By DSENL, the discriminability of the learned hash codes is improved. Experimental results on four benchmark datasets demonstrate the effectiveness of DSENL. The source code is available at https://github.com/niuniubit/DSENL.git.