Multi-Modal Semantic Enhancement for Early Fake News Detection

In recent years, with the increasing popularity of social media, one piece of news widely spread after it was released, which can cause negative effects on social life and make it a severe problem. Existing automatic fake news detection methods fail to consider the semantic consistency and complement among different types of features. The paper proposes an early fake news detection method based on multi-model semantic enhancement, which can fully explore the deep interaction relationship between multi-modal semantic features. First, the proposed model combines Bidirectional Encoder Representation from Transformers (BERT) with a Bidirectional Long Short-Term Memory (BiLSTM) network and Residual Network (ResNet) to extract deep semantic features of text and images, respectively. Furthermore, we employ the co-attention mechanism to fuse multi-model features, which better estimates the semantic consistency and learns the semantic interaction features between text and images. Then, the fused multi-modal feature is re-fused with original features to strengthen the role of original information, obtaining the final multi-modal representation for fake news detection. Extensive experiments conducted on the Weibo dataset show that the best accuracy is 94.21%, and the F1-score is 94.36%, outperforming other baseline models.