Dynamic TCP Initial Windows and Congestion Control Schemes Through Reinforcement Learning

Despite many years of improvements to it, TCP still suffers from an unsatisfactory performance. For services dominated by short flows (e.g., web search and e-commerce), TCP suffers from the flow startup problem and cannot fully utilize the available bandwidth in the modern Internet: TCP starts from...

Full description

Saved in:
Bibliographic Details
Published in:IEEE journal on selected areas in communications 2019-06, Vol.37 (6), p.1231-1247
Main Authors: Nie, Xiaohui, Zhao, Youjian, Li, Zhihan, Chen, Guo, Sui, Kaixin, Zhang, Jiyang, Ye, Zijie, Pei, Dan
Format: Article
Language:English
Subjects:
Bandwidth
Congestion
congestion control
Data communication
Digital media
Machine learning
Microsoft Windows
Performance enhancement
Reinforcement learning
Servers
TCP (protocol)
TCP initial window
TCP-IP
Video transmission
web service
Web services
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Despite many years of improvements to it, TCP still suffers from an unsatisfactory performance. For services dominated by short flows (e.g., web search and e-commerce), TCP suffers from the flow startup problem and cannot fully utilize the available bandwidth in the modern Internet: TCP starts from a conservative and static initial window ( IW , 2-4 or 10), while most of the web flows are too short to converge to the best sending rate before the session ends. For services dominated by long flows (e.g., video streaming and file downloading), the congestion control ( CC ) scheme manually and statically configured might not offer the best performance for the latest network conditions. To address these two challenges, we propose TCP-RL , which uses reinforcement learning ( RL ) techniques to dynamically configure IW and CC in order to improve the performance of TCP flow transmission. Basing on the latest network conditions observed at the server side of a web service, TCP-RL dynamically configures a suitable IW for short flows through group-based RL , and dynamically configures a suitable CC scheme for long flows through deep RL . Our extensive experiments show that for short flows, TCP-RL can reduce the average transmission time by about 23%; and for long flows, compared with the performance of 14 CC schemes, TCP-RL 's performance ranks top 5 for about 85% of the 288 given static network conditions, whereas for about 90% of conditions, its performance drops by less than 12% compared with that of the best-performing CC schemes for the same network conditions.
ISSN:0733-8716
1558-0008
DOI:10.1109/JSAC.2019.2904350