A Relationship Extraction Model Based on Reinforcement Learning and Multi-Teacher Knowledge Distillation

Pre-trained models significantly enhance the effectiveness of relation extraction tasks, but the high computational costs associated with these models pose challenges for deployment. This paper introduces a dynamic relation extraction model based on multi-teacher knowledge distillation. The model jointly trains the student model with features from the intermediate and output layers of teacher models. The training progress of the student model is evaluated, and a reinforcement learning approach is employed to optimize the selection of teacher models based on the assessment results of the student model. Subsequently, this optimized scheme guides the training of the student model. Experimental results demonstrate that the proposed method achieves the most substantial improvement in the performance of the student model.