Spatiotemporal and region relationship network for micro-expression recognition

Micro-expressions are subtle facial expressions that occur when a person fails to suppress his emotional response. A significant issue is the intricate correlation between micro-expression motion and various facial regions, which hinders the extraction of effective feature. This paper proposes a novel spatiotemporal and regional relationship network (STRNet) to address this issue. STRNet consists of two branches and a specialised feature fusion module. Specifically, one branch performs multi-level feature extraction. The other branch focuses on modelling the relationships between different facial regions. The outputs of the two branches are then combined through a feature fusion module; this module enhances the generalisation of the micro-expression features extracted by the network. Experiments on four public datasets (SAMM, CASMEII, SMIC and CASME3) validate STRNet's effectiveness. On CASMEII, it achieved UF1 and UAR scores of 0.9792 and 0.9764, respectively, and on CASME3, 0.5848 and 0.5601. STRNet outperformed existing methods, demonstrating superior performance.