SENTIMENT ANALYSIS MODEL BASED ON ATTENTION MECHANISM MULTIMODAL FUSION AND SPATIO-TEMPORAL GRAPH NEURAL NETWORK

As a key task in the field of artificial intelligence, the application of sentiment analysis has expanded from a single text modality to cover multi-modal information such as vision and hearing. However, the existing multi-modal sentiment analysis methods often ignore the time sequence and structure of the dynamic interaction between modalities, and fail to fully model the evolution of emotional States in the time dimension. Therefore, this study proposes a sentiment analysis model based on multi-modal fusion of attention mechanism and spatio-temporal neural network. The model first extracts the high-dimensional features of text, visual and auditory modalities through a dedicated encoder, and then designs a hierarchical attention fusion mechanism to adaptively weight the contributions of different modalities at the feature level and decision level. In order to capture the dynamic evolution of emotional expression, the spatio-temporal graph neural network module is introduced into the model, and the fusion features of each time step are regarded as graph nodes, and the dynamic edges are constructed based on the correlation and temporal continuity between modalities, so as to model the spatio-temporal dependence of cross-modal interaction. Experiments are conducted on three open multimodal emotion datasets, including CMU-MOSEI, IEMOCAP and MOSI. The results show that the proposed model significantly outperforms the baseline method in both sentiment classification and sentiment regression tasks. On the CMU-MOSEI data set, the accuracy of the model reaches 88. 7% in the binary classification and 53. 2% in the seven-class classification, both of which reach the current advanced level. The ablation experiment further verifies the effectiveness of the hierarchical attention mechanism and the spatiotemporal neural network module. This study provides a new framework for multimodal sentiment analysis that can simultaneously model the complex interaction and temporal dynamics between modalities, which has theoretical significance for understanding the complex mechanism of human emotional expression, and provides technical support for the development of more accurate emotional intelligence applications.