Profiling transport performance for big data transfer over dedicated channels

The transfer of big data is increasingly supported by dedicated channels in high-performance networks. Transport protocols play a critical role in maximizing the link utilization of such high-speed connections. We propose a Transport Profile Generator (TPG) to characterize and enhance the end-to-end...

Full description

Saved in:
Bibliographic Details
Main Authors: Daqing Yun, Wu, Chase Q., Rao, Nageswara S. V., Settlemyer, Bradley W., Lothian, Josh, Kettimuthu, Rajkumar, Vishwanath, Venkatram
Format: Conference Proceeding
Language:English
Subjects:
Bandwidth
big data transfer
Data transfer
high-performance networks
Optical buffering
Servers
Sockets
Throughput
Transport profiling
Transport protocols
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The transfer of big data is increasingly supported by dedicated channels in high-performance networks. Transport protocols play a critical role in maximizing the link utilization of such high-speed connections. We propose a Transport Profile Generator (TPG) to characterize and enhance the end-to-end throughput performance of transport protocols. TPG automates the tuning of various transport-related parameters including socket options and protocol-specific configurations, and supports multiple data streams and multiple NIC-to-NIC connections. To instantiate the design of TPG, we use UDT as an example in the implementation and conduct extensive experiments of big data transfer over high-speed network channels to illustrate how existing transport protocols benefit from TPG in optimizing their performance.
DOI:10.1109/ICCNC.2015.7069458