A Lightweight Communication Runtime for Distributed Graph Analytics

• Main Authors: Dang, Hoang-Vu, Dathathri, Roshan, Gill, Gurbinder, Brooks, Alex, Dryden, Nikoli, Lenharth, Andrew, Hoang, Loc, Pingali, Keshav, Snir, Marc
• Format: Conference Proceeding
• Language: English
• Subjects: Benchmark testing; clusters; communication systems; Graph analytics; Message systems; Mirrors; Probes; Runtime; Semantics; Synchronization
• ISSN: 1530-2075
• DOI: 10.1109/IPDPS.2018.00107

Distributed-memory multi-core clusters enable in-memory processing of very large graphs with billions of nodes and edges. Recent distributed graph analytics systems have been built on top of MPI. However, communication in graph applications is very irregular, and each host exchanges different amounts of non-contiguous data with other hosts. MPI does not support such a communication pattern well, and it has limited ability to integrate communication with serialization, deserialization, and graph computation tasks. In this paper, we describe a lightweight communication runtime called LCI that supports a large number of threads on each host and avoids the semantic mismatches between the requirements of graph computations and the communication library in MPI. The implementation of LCI is informed by lessons learnt from two baseline MPI-based implementations. We have successfully integrated LCI with two state-of-the-art graph analytics systems - Gemini and Abelian. LCI improves the latency up to 3.5× for microbenchmarks compared to MPI solutions and improves the end-to-end performance of distributed graph algorithms by up to 2×.