Graph Attention Networks

The attention mechanism has been successfully used in many sequence-based tasks such as machine translation [Bahdanau et al., 2015, Gehring et al., 2017, Vaswani et al., 2017], machine reading [Cheng et al., 2016], and so on. Compared with GCN which treats all neighbors of a node equally, the attention mechanism could assign different attention score to each neighbor, thus identifying more important neighbors. It is intuitive to incorporate the attention mechanism into the propagation steps of Graph Neural Networks. In this chapter, we will talk about two variants: GAT and GAAN.