Multimodal Sentiment Analysis—A Comprehensive Survey From a Fusion Methods Perspective

Multimodal sentiment analysis (MSA) has emerged as a crucial domain within artificial intelligence, aimed at accurately interpreting human emotions by integrating information from multiple data modalities such as text, audio, and visual signals. Traditional sentiment analysis (SA) predominantly focused on unimodal data, restricting the capability to interpret subtle emotional expressions. Leveraging the progress in machine learning and deep learning, multimodal fusion plays a crucial role in harnessing the complementary features of diverse modalities, thereby improving SA performance. This paper provides a comprehensive survey from a perspective of fusion methods, encompassing early fusion, late fusion, hybrid fusion, tensor fusion, contextual fusion, hierarchical fusion, graph fusion and attention mechanism-based approaches. We also present detailed statistics on popular datasets. We categorize three types of feature extraction techniques critical for MSA. Furthermore, this work emphasizes key challenges, including figurative language recognition, dataset biases, dataset preparation, intermodal imbalance, noise removal, modalities missing, synchronization issues, increased dimensionality and selection of fusion method. Through an in-depth exploration of these issues, we highlight future directions for advancing fusion methodologies and propose strategies for developing more robust, adaptive, and generalizable models. This paper provides valuable insights serving as a valuable resource for practitioners and researchers, guiding the development of innovative solutions for MSA across diverse application domains.