Determining the Optimal Redistribution for a Given Data Partition

The classical redistribution problem aims at optimally scheduling communications when moving from an initial data distribution to a target distribution where each processor will host a subset of data items. However, modern computing platforms are equipped with a powerful interconnection switch, and...

Full description

Saved in:
Bibliographic Details
Main Authors: Herault, Thomas, Herrmann, Julien, Marchal, Loris, Robert, Yves
Format: Conference Proceeding
Language:English
Subjects:
Bipartite graph
Complexity theory
Distributed databases
Kernel
Measurement
Optimization
Partitioning algorithms
Citations: Items that cite this one
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The classical redistribution problem aims at optimally scheduling communications when moving from an initial data distribution to a target distribution where each processor will host a subset of data items. However, modern computing platforms are equipped with a powerful interconnection switch, and the cost of a given communication is (almost) independent of the location of its sender and receiver. This leads to generalizing the redistribution problem as follows: find the optimal one-toone mapping of the subsets of data items onto the processors for which the cost of the redistribution is minimal. This paper studies the complexity of this generalized problem. We provide optimal algorithms and evaluate their gain over classical redistribution through simulations. We also show the NP-hardness of the problem to find the optimal data partition and processor permutation (defined by new subsets) that minimize the cost of redistribution followed by a simple computation kernel.
ISSN:2379-5352
DOI:10.1109/ISPDC.2014.16