GANK: Dynamic Geometric and Appearance Features for Efficient and Robust Detection of Face Forgery

Deepfakes refers to various deep-learning-based techniques that manipulate the face in videos. Maliciously manufactured face forgeries could result in serious problems such as portrait infringement, information confusion, or even public panic. Previous countermeasures focused mainly on promoting detection accuracy while relatively overlooking robustness and computational overhead. In this work, we propose an efficient and robust framework named GANK , which discriminates Deepfake videos through temporal modeling on decoupled geometric and appearance features. A temporal denoising technique featuring landmark tracking and Kalman filtering is introduced to optimize the feature sequences, and multi-stream Recurrent Neural Networks (RNN) are constructed for sufficient exploitation of dynamic features. Besides, we introduce two optimizations to alleviate overfitting and enhance the utilization of temporal information, including channel-wise dropout and temporal random cropping. Our framework achieves outstanding robustness using very lightweight network backbones, reaching the state-of-the-art performance on multiple benchmarks.