CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation

Multimodal sentiment analysis and depression estimation are two important\nresearch topics that aim to predict human mental states using multimodal data.\nPrevious research has focused on developing effective fusion strategies for\nexchanging and integrating mind-related information from different modalities.\nSome MLP-based techniques have recently achieved considerable success in a\nvariety of computer vision tasks. Inspired by this, we explore multimodal\napproaches with a feature-mixing perspective in this study. To this end, we\nintroduce CubeMLP, a multimodal feature processing framework based entirely on\nMLP. CubeMLP consists of three independent MLP units, each of which has two\naffine transformations. CubeMLP accepts all relevant modality features as input\nand mixes them across three axes. After extracting the characteristics using\nCubeMLP, the mixed multimodal features are flattened for task predictions. Our\nexperiments are conducted on sentiment analysis datasets: CMU-MOSI and\nCMU-MOSEI, and depression estimation dataset: AVEC2019. The results show that\nCubeMLP can achieve state-of-the-art performance with a much lower computing\ncost.\n