Facial Expression Recognition in-the-Wild Using Blended Feature Attention Network

Facial expression analysis plays a crucial role in various fields, such as affective computing, marketing, and clinical evaluation. Despite numerous advances, research on facial expression recognition (FER) has recently been proceeding from confined lab circumstances to in-the-wild environments. FER is still an arduous and demanding problem due to occlusion and pose changes, intra-class and intensity variations caused by illumination, and insufficient training data. Most state-of-the-art approaches use entire face for FER. However, the past studies on psychology and physiology reveals that mouth and eyes reflect the variations of various facial expressions, which are closely related to the manifestation of emotion. A novel method is proposed in this study to address some of the issues mentioned above. Firstly, modified homomorphic filtering is employed to normalize the illumination, then the normalized face image is cropped into five local regions to emphasize expression-specific characteristics. Finally, a unique blended feature attention network (BFAN) is designed for FER. BFAN consists of both residual dilated multi-scale feature extraction module and spatial and channel-wise attention modules. These modules help to extract the most relevant and discriminative features from the high-level and low-level features. Then, both feature maps are integrated and passed on to the dense layers followed by a softmax layer to compute probability scores. Finally, the Choquet fuzzy integral is applied to the computed probability scores to get the final outcome. The superiority of the proposed method is exemplified by comparing it with eighteen existing approaches on seven benchmark datasets.