Gator: Accelerating Graph Attention Networks by Jointly Optimizing Attention and Graph Processing

Graph attention networks (GATs) have advanced performance in various application domains by introducing the attention mechanism into the graph neural networks (GNNs). The inefficiency of running GATs on CPUs or GPUs necessitates specialized hardware designs. Unfortunately, previous specialized architecture designs have focused on either the GNN architecture or the attention mechanism, resulting in limited performance and leaving ample room for improvement. This paper presents Gator , a joint optimization approach with software-hardware co-designs for GAT inference. On the software level, Gator leverages degree-weighted graph partitioning and parameter-adaptive feature selection techniques to preprocess the input graph data, mining subgraph-level parallelism and mitigating the computation bottleneck of the dedicated dataflow. On the hardware level, Gator designs a unified processing engine to support various kernels by extracting a common computation pattern, and a dimension-aware microarchitecture for efficient partial sum reduction. Extensive experiments show that our approach can achieve 11.5 × more efficiency compared to NVIDIA RTX 4090, and provide a speedup of 3 × to 9.4 ×, along with a 2.6 × to 4.7 × reduction in memory traffic when compared to six state-of-the-art methods, with minimal accuracy loss.