A Retrieve-and-Read Framework for Knowledge Graph Link Prediction

Knowledge graph (KG) link prediction aims to infer new facts based on\nexisting facts in the KG. Recent studies have shown that using the graph\nneighborhood of a node via graph neural networks (GNNs) provides more useful\ninformation compared to just using the query information. Conventional GNNs for\nKG link prediction follow the standard message-passing paradigm on the entire\nKG, which leads to superfluous computation, over-smoothing of node\nrepresentations, and also limits their expressive power. On a large scale, it\nbecomes computationally expensive to aggregate useful information from the\nentire KG for inference. To address the limitations of existing KG link\nprediction frameworks, we propose a novel retrieve-and-read framework, which\nfirst retrieves a relevant subgraph context for the query and then jointly\nreasons over the context and the query with a high-capacity reader. As part of\nour exemplar instantiation for the new framework, we propose a novel\nTransformer-based GNN as the reader, which incorporates graph-based attention\nstructure and cross-attention between query and context for deep fusion. This\nsimple yet effective design enables the model to focus on salient context\ninformation relevant to the query. Empirical results on two standard KG link\nprediction datasets demonstrate the competitive performance of the proposed\nmethod. Furthermore, our analysis yields valuable insights for designing\nimproved retrievers within the framework.\n