SDHF: Spotting DeepFakes with Hierarchical Features
Tao Liang, Peng Chen, Guangzhi Zhou, Hongchao Gao, Jin Liu, Zhaoxing Li, Jiao Dai
Abstract
DeepFake videos are widely distributed on social media platforms, which has seriously affected the authenticity of digital media content, calling for robust DeepFake detection methods. Although numerous detection methods are formulated as frame-based binary classification, less attention has been paid to aggregate the features over individual frames to get a video-based judgement. We observed that for the detection of DeepFake videos, three different level forgery features from frame, clip and video can complement each other. We also found that discrete, large interval sampling strategy is more suitable for DeepFake detection, which can sample more complex video scenes, including multiple subjects, diverse facial expressions and head poses. In this work, we propose a hierarchical framework, using 2D convolutional neural networks for frame-level features extraction followed by a 1D convolutional aggregator to extract clip-level and video-level features, which can comprehensively exploit three different levels of features to make decisions. Evaluation was performed on four datasets, including DFDC, Celeb-DF, FaceForensics++ and UADFV, which provides competitive results compared to other methods. Experimental results of cross-test demonstrate that our hierarchical framework has excellent generalization performance in the face of unknown datasets.