Two-Stage Knowledge Distillation Network With Multi-Teacher Self-Training for Semi-Supervised Multi-Modal Fake News Detection

Acquiring news on social media has become the main way for people to stay informed, making it crucial to verify the credibility of news on social media. With the diversification of social media, news is often composed of multi-modal data. Additionally, in practical applications, we face the challenge of owning labels for only a small set of news data, with the majority of news data remaining unlabeled due to the substantial labeling cost. We define the fake news detection problem under this scenario as semi-supervised multi-modal fake news detection (SMFND). However, current SMFND methods do not fully explore the semantic and structural information in unlabeled data. We propose a two-stage knowledge distillation network with multi-teacher self-training (TKDN) method for SMFND, consisting of a semantic and structural knowledge distillation (SSKD) module, a joint supervision (JS) module, and a self-training based teacher model updating (STMU) mechanism. Firstly, we construct two teacher models and one student model with the same network structure but different initialization parameters. The SSKD module leverages multiple teacher models to transfer both multi-modal semantic and adjacency structure knowledge to the student model, effectively enriching knowledge learned by the student model. The JS module provides high-quality pseudo-labels to the student model through multi-teacher model collaboration, improving identification ability and accelerating the convergence of the student model. Finally, we pre-train the teacher model using the STMU mechanism with high-quality pseudo-labels to enhance the reliability of teacher model. Experiments on three widely used fake news datasets have shown that our method outperforms state-of-the-art related works.