IOCost: Block Input–Output Control for Containers in Datacenters

• Published in: IEEE MICRO 2023-07, Vol.43 (4), p.80-87
• Main Authors: Heo, Tejun, Schatzberg, Dan, Newell, Andrew, Liu, Song, Dhakshinamurthy, Saravanan, Narayanan, Iyswarya, Bacik, Josef, Mason, Chris, Tang, Chunqiang, Skarlatos, Dimitrios
• Format: Article
• Language: English
• Subjects: Algorithms; Budgets; Containers; Data centers; Workload
• ISSN: 0272-1732; 1937-4143
• DOI: 10.1109/MM.2023.3277783

Resource isolation is a requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing input–output (IO) control mechanisms for block storage are inadequate in containerized environments. This article presents IOCost, an IO control solution designed for containerized environments that provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across Meta’s datacenters comprising millions of machines, upstreamed IOCost to the Linux kernel, and open sourced our device-profiling tools.