OTRec: Cross-Modal Learning for Multimodal Recommendation via Optimal Transport

In recent years, there has been a growing interest in multimodal recommendation systems due to the rapid growth of multimedia and the explosion of information. Despite notable advancements, current models often fuse multimodal embeddings with ID (name or concept) embeddings in a weighted or concatenated manner for items. Under this circumstance, they may overlook the heterogeneity problem between different modalities, and lack theoretical guarantees, potentially leading to suboptimal item representations. To overcome this challenge, we introduce a novel model named OTRec, which employs optimal transport (OT) to align heterogeneous multimodal embeddings with ID embeddings. Specifically, OTRec captures co-occurrence features across modalities and distinctive features within modalities, enabling the formation of the unified representation from both modalinvariant and modal-specific perspectives. This dual strategy ensures a comprehensive alignment of heterogeneous multimodal data, significantly improving the accuracy of capturing user preferences. Additionally, traditional recommendation models typically match an item's ID with its multimodal data as positive samples for contrastive learning, neglecting the potential complementary information from other items' multimodal data. To address this issue, we introduce a semanticenhanced contrastive learning module, which can learn latent semantic correlations across items by a semantic-similarity weighting matrix. It can be integrated as a plug-in for other models to effectively explore latent semantics. On top of this, we provide theoretical guarantees that demonstrate the effectiveness of OTRec in aligning multimodal and ID information and in enhancing the mutual information between them. Extensive evaluations on three public datasets illustrate OTRec's effectiveness and achieve state-of-the-art performance.