Label-Dependent Graph Neural Network

Graph neural network (GNN) provides a powerful expressive way to embed graph-structured data, which has been widely applied and spans many fields. This article found an interesting but unreasonable phenomenon in some classical GNNs, namely label confusion. Theoretically, the dependence between prediction label and ground-truth should gradually increase and tend to converge with training epochs. On the contrary, label confusion shows that the dependence between the predicted label and ground-truth is strong first and then weak. This article explains it from the perspective of feature-noise dependence. Specifically, traditional GNN prediction usually assumes that the ground-truth consists of a mutually independent prediction label (dependent on node features) and noise (independent of node features). However, the propagation aggregation mechanism of GNN will integrate irrelevant information from neighboring nodes into the prediction label, resulting in the prediction label no longer being completely dependent on the node feature, or the noise no longer being completely independent of the node feature. Hence, this article proposes a Label-Dependent GNN to alleviate this problem, called LDGNN. LDGNN mainly consists of two limitations, namely feature-noise (the difference between predicted label and ground-truth) independence, and expectation-variance (EV) separation. Specifically, LDGNN introduces the Hilbert-Schmidt independence criterion (HSIC) as a regularization to minimize the dependence between input features and noise. Note that the main reason for adopting HSIC is that it can measure the nonlinear relationship between any two spatial variables. In this way, HSIC can guide GNN to retain more label-dependence information. Then, LDGNN designs an EV separation to centralize nodes within a class and disperse them between classes to further retain label-dependence information. Through these two strategies, the GNN's expression ability can be enhanced. Next, we theoretically prove the essential reason why LDGNN alleviates label confusion and has been verified in experiments. To verify the performance of LDGNN, we apply it to four classical GNN models on three datasets, and experimental results demonstrate the effectiveness of LDGNN.