Node-Level Graph Regression With Deep Gaussian Process Models

In this paper, we study node-level graph regression, which aims to predict an output vector for each node on a given graph. This task has a broad range of applications, including spatio-temporal forecasting and computational biology. We propose a model called Deep Gaussian Processes over Graphs (DGPG), which is composed of hierarchical Gaussian processes and learns the mapping between input-output signals in graph domains. DGPG possesses several distinctive advantages, such as the ability in capturing uncertainty, effectiveness on small datasets, and requiring fewer efforts for selecting model architectures and hyperparameters. It is also more favorable than traditional Gaussian process models in terms of expressiveness and scalability, due to the hierarchical deep structure and the variational inference framework. Moreover, we generalize DGPG to a more challenging setting where the graph structure is time-varying. Our theoretical analysis shows that the graph information can improve convergence by reducing sampling variances when optimizing the evidence lower bound, and the challenge of time-varying graph structure can be addressed by a time-weighted sampling scheme. The performance of DGPG is demonstrated through extensive experiments in various synthetic and real-world datasets. Some appealing characteristics of DGPG are further discussed, such as its ability to capture prediction uncertainty and learn graph structures.