The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: Workshop on Machine Learning with Graphs in High Performance Computing Environments
Authors: Gangda Deng, Ömer Akgül, and Hongkuan Zhou (University of Southern California (USC)); Hanqing Zeng, Yinglong Xia, and Jianbo Li (Meta); and Viktor Prasanna (University of Southern California (USC))
Abstract: Traditional graph-processing algorithms have been widely used in Graph Neural Networks (GNNs). Current approaches to graph processing in deep learning face two main problems. Firstly, easy-to-use deep learning libraries lack support for widely used graph processing algorithms and do not provide low-level APIs for building distributed graph processing algorithms. Secondly, existing graph processing libraries are not user-friendly for deep learning researchers. This paper presents an efficient and easy-to-use graph engine that incorporates distributed graph processing into deep-learning ecosystems. We develop a distributed graph storage system with an efficient batching technique to minimize communication overhead incurred by Remote Procedure Calls between computing nodes. We propose an optimized method for distributed computation of Single Source Personalized PageRank (SSPPR) using the Forward Push algorithm based on lock-free parallel maps. Experimental evaluations demonstrate significant improvement, with up to three orders of magnitude in SSPPR throughput, of our graph engine compared with the tensor-based implementation.