Patch Attention Network for Video Facial Expression Recognition

Facial expression recognition (FER) is a hot research topic in computer vision. In recent years, attention mechanism has been widely used in facial expression recognition tasks and achieved good results. However, most methods applying attention mechanism are aimed at static image-based FER. In this paper, we propose a novel convolution neutral network (CNN) with attention mechanism for video-based FER. Our network introduces Patch Diff Attention (PDA) module to focus on regions with large variation, and Patch Self Attention (PSA) module to focus on regions containing more expression information. With extensive experiments on CK+ and AFEW datasets, our proposed method shows superior or similar performance compared to the state-of-the-art approaches.