AIGC video detection based on the fusion of spatial-frequency-optical flow multimodal features
Hong Sheng, Wang Xuanqi, Zhang Chang, Wang Jiacheng, Duan Pingxia, Wang Yuwei
Abstract
The rapid evolution of generative artificial intelligence (AI) (e.g., Sora, Hunyuan) makes it essential to develop effective detection strategies that can generalize across ever-evolving synthesis techniques. This study is motivated by the observation of a fundamental challenge in generative models: the inherent difficulty of maintaining cross-modal consistency between appearance and motion. To this end, we propose a multi-modal framework for AI generated content (AIGC) video forgery detection tasks, named cross-attention based video forgery detector (CrossAtt-VFD), based on joint multi-view analysis of content. Methodologically, we introduce a dual-branch architecture that simultaneously extracts spatial-frequency and optical-flow features. This approach enables the modeling of videos from complementary perceptual perspectives. The core of this process is a dedicated cross-attention mechanism, which governs the alignment of the two modalities and translates cross-modal inconsistencies into a potent diagnostic signal. This multi-modal strategy facilitates the detection of motion that is statistically inconsistent with the visual appearance of a scene. Comprehensive experimental results demonstrate that our model achieves an accuracy of 94.32%, a precision of 91.67%, and a recall of 96.25%, effectively verifying the advantages of the multi-modal fusion strategy.