Multimodal Cheapfakes Detection by Utilizing Image Captioning for Global Context
Tuan-Vinh La, Quang-Tien Tran, Thanh-Phuc Tran, Anh-Duy Tran, Duc‐Tien Dang‐Nguyen, Minh-Son Dao
Abstract
The rapid development of technology in social media platforms has led to abundant misinformation and fake news spreading in the community. One of the most prevalent ways to misleading information on social media is cheapfakes, which are more accessible and affordable than deepfakes. Most existing approaches extract features from text or concatenate visual and textual features and train with multimodal to classify news. This paper proposed several strategies to leverage object, textual, image captioning features. These strategies focus on utilizing image captioning to extract the correlation between images and captions. We also propose some boosting techniques to enhance the result. Our methods are evaluated on the "MMSys'21 Grand Challenge" dataset and have 86.75% accuracy.