Hierarchical Graph Learning Framework for Multimodal Conversational Emotion Recognition

Accurate emotion detection in conversations using multimodal features is essential for effective human-computer interaction. There are three pivotal aspects in multimodal emotion recognition in conversation (MERC), i.e., intricate temporal information, modality interactions (both intra- and inter-modal) and implicit high-order linguistic cues within dialogues. Existing approaches are limited to the first two, hindering the generation of effective emotional representations. To this end, we propose HIGH, a hierarchical graph learning framework designed for MERC. HIGH enhances the perception of low-level information (temporal and intra-modal information) by constructing directed dialogue graphs for each modality. A dynamic multimodal filtering mechanism and a modality-aligned contrastive learning approach further refine the semantic nuances. Additionally, the constructed speaker-centered hypergraph yields high-level information like cross-modal interactions and high-order linguistic cues between utterances. HIGH effectively integrates fundamental low-level information with high-order details in a hierarchical manner, considering all three key factors simultaneously. Extensive experiments on two benchmark datasets demonstrate the effectiveness and superiority of HIGH.