Technical Papers Archive

TANGO: Re-Thinking Quantization for Graph Neural Network Training on GPUs

Authors: Shiyang Chen (Rutgers University); Da Zheng (Amazon); Caiwen Ding (University of Connecticut); Chengying Huan (Institute of Software, Chinese Academy of Sciences); Yuede Ji (University of North Texas); and Hang Liu (Rutgers University)

Abstract: Graph Neural Networks (GNNs) are rapidly gaining popularity since they hold state-of-the-art performance for various critical graph-related tasks. While quantization is a primary approach to accelerating GNN computation, quantized training faces remarkable challenges. We observe that current quantized GNN training systems often experience longer training time than their full-precision counterparts for two reasons: (i) addressing the accuracy challenge results in too much overhead. (ii) The optimization opportunity exposed by quantization is not well leveraged. This paper introduces Tango, which re-thinks quantization challenges and opportunities for graph neural network training on GPUs with the following contributions: First, we introduce light-weighted rules to meet the accuracy requirement for quantized GNN training. Second, we design and implement quantization-aware primitives and inter-primitive optimizations to accelerate GNN training. Third, we integrate Tango with the mainstream Deep Graph Library (DGL) system and demonstrate that Tango outperforms the state-of-the-art across all the evaluated GNN models and datasets.




Back to Technical Papers Archive Listing