Application-driven graph partitioning

Graph partitioning is crucial to parallel computations on large graphs. The choice of partitioning strategies has strong impact on the performance of graph algorithms. For an algorithm of our interest, what partitioning strategy fits it the best and improves its parallel execution? Is it possible to...

Full description

Saved in:
Bibliographic Details
Published in:The VLDB journal 2023, Vol.32 (1), p.149-172
Main Authors: Fan, Wenfei, Xu, Ruiqi, Yin, Qiang, Yu, Wenyuan, Zhou, Jingren
Format: Article
Language:English
Subjects:
Algorithms
Computer Science
Database Management
Graphs
Partitioning
Performance enhancement
Polynomials
Regular Paper
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Graph partitioning is crucial to parallel computations on large graphs. The choice of partitioning strategies has strong impact on the performance of graph algorithms. For an algorithm of our interest, what partitioning strategy fits it the best and improves its parallel execution? Is it possible to provide a uniform partition to a batch of algorithms that run on the same graph simultaneously, and speed up each and every of them? This paper aims to answer these questions. We propose an application-driven hybrid partitioning strategy that, given a graph algorithm A , learns a cost model for A as polynomial regression. We develop partitioners that, given the learned cost model, refine an edge-cut or vertex-cut partition to a hybrid partition and reduce the parallel cost of A . Moreover, we extend the cost-driven strategy to support multiple algorithms at the same time and reduce the parallel cost of each of them. Using real-life and synthetic graphs, we experimentally verify that our partitioning strategy improves the performance of a variety of graph algorithms, up to 22.5 × .
ISSN:1066-8888
0949-877X
DOI:10.1007/s00778-022-00736-2