Enhancing Conversational AI with Emotion Recognition: A Multimodal Approach

The state-of-the-art emotion recognition in conversational AI has attracted considerable attention, as it has the potential to improve user experience through adaptive itineraries and empathetic interactions. The paper provides a comprehensive overview of the current state-of-the-art, including breakthroughs, hurdles, and approaches towards recognizing emotion in conversational agents. The initial rule-based methods were based on structure maps and predefined lexicons, with recent classifier-based and graph-based methods providing accurate methods to detect emotions, based on deep learning techniques, such as BERT and GPT transformer-based methods. Furthermore, multimodal techniques that jointly leverage textual, speech, and visual signals have shown better effectiveness in detecting fine-grained emotion expressions. Nevertheless, issues like cultural differences in emotion expression, vague user inputs, and privacy concern remain. We review current techniques, practical limitations and future developments on emotion recognition systems for conversational AI. Overcoming these challenges will result in better human-computer interaction with AI systems that are natural, context-aware and empathetic.