Multimodal contrastive learning for face anti-spoofing

Multimodal face anti-spoofing systems adopt multiple sensor modalities, such as infrared, color, depth, and thermal, to distinguish between living and spoofing faces via complementary spoofing clues from each modality. One challenge is that when the multimodal face anti-spoofing system is placed in different environments, the sensor setup may not be unified, causing a certain sensor to be unavailable. To alleviate this issue, a two-stream face anti-spoofing method is proposed. The first stream focuses on extracting primary features from an available sensor by a baseline network. The second stream employs a multimodal contrastive learning strategy to acquire modality-agnostic and task-specific representations from another deployed sensor. Furthermore, a master–slave modulation fusion block is designed to effectively fuse features from the two streams. Experiments conducted on three public multimodal databases show the superior performance of the proposed method.