Exoshuffle-CloudSort

We present Exoshuffle-CloudSort, a sorting application running on top of Ray using the Exoshuffle architecture. Exoshuffle-CloudSort runs on Amazon EC2, with input and output data stored on Amazon S3. Using 40 i4i.4xlarge workers, Exoshuffle-CloudSort completes the 100 TB CloudSort Benchmark (Indy c...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-01
Main Authors: Luan, Frank Sifei, Wang, Stephanie, Yagati, Samyukta, Kim, Sean, Lien, Kenneth, Ong, Isaac, Hong, Tony, Cho, SangBin, Liang, Eric, Stoica, Ion
Format: Article
Language:English
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present Exoshuffle-CloudSort, a sorting application running on top of Ray using the Exoshuffle architecture. Exoshuffle-CloudSort runs on Amazon EC2, with input and output data stored on Amazon S3. Using 40 i4i.4xlarge workers, Exoshuffle-CloudSort completes the 100 TB CloudSort Benchmark (Indy category) in 5378 seconds, with an average total cost of $97.
ISSN:2331-8422