A Knowledge Distillation-Based Approach for Dynamic Graph Representation Learning

Dynamic graphs are prevalent data structures in the real world, providing powerful modeling capabilities. However, popular methods for dynamic graph representation learning often rely on graph data. Models typically require input of the entire graph's adjacency matrix and aggregate information based on node neighborhoods. This leads to challenges in model deployment and slow inference speeds. This paper combines knowledge distillation with dynamic graph representation learning, transferring graph-related knowledge learned by the teacher model to a student model. The link prediction experimental results demonstrate that introducing knowledge distillation yields a well-performing and deployable student model comparable to the teacher model. Additionally, the student model overcomes its reliance on graph data, eliminating the process of aggregating node neighborhood information through the input adjacency matrix, thereby improving inference speed.