Facial expression and body gesture emotion recognition: A systematic review on the use of visual data in affective computing

Emotion is an important driver of human decision-making and communication. With the recent rise of human–computer interaction, affective computing has become a trending research topic, aiming to develop computational systems that can understand human emotions and respond to them. A systematic review has been conducted to fill these gaps since previous reviews regarding machine-enabled automated visual emotion recognition neglect important methodological aspects, including emotion models and hardware usage. 467 relevant papers were initially found and examined. After the screening process with specific inclusion and exclusion criteria, 30 papers were selected. Methodological aspects including emotion models, devices, architectures, and classification techniques employed by the selected studies were analyzed, and the most popular techniques and current trends in visual emotion recognition were identified. This review not only offers a comprehensive and up-to-date overview of the topic but also provides researchers with insights regarding methodological aspects like emotion models employed, devices used, and classification techniques for automated visual emotion recognition. By identifying current trends, like the increased use of deep learning algorithms and the need for further study on body gestures, this review advocates the advantages of implementing emotion recognition with the use of visual data and builds a solid foundation for applying relevant techniques in different fields.