DGSF: Disaggregated GPUs for Serverless Functions

• Main Authors: Fingler, Henrique, Zhu, Zhiting, Yoon, Esther, Jia, Zhipeng, Witchel, Emmett, Rossbach, Christopher J.
• Format: Conference Proceeding
• Language: English
• Subjects: API remoting; Cloud computing; Distributed processing; FaaS; GPU; Graphics processing units; Machine learning; Production; Prototypes; Runtime; serverless
• ISSN: 1530-2075
• DOI: 10.1109/IPDPS53621.2022.00077

Ease of use and transparent access to elastic resources have attracted many applications away from traditional platforms toward serverless functions. Many of these applications, such as machine learning, could benefit significantly from GPU acceleration. Unfortunately, GPUs remain inaccessible from serverless functions in modern production settings. We present DGSF, a platform that transparently enables serverless functions to use GPUs through general purpose APIs such as CUDA. DGSF solves provisioning and utilization challenges with disaggregation, serving the needs of a potentially large number of functions through virtual GPUs backed by a small pool of physical GPUs on dedicated servers. Disaggregation allows the provider to decouple GPU provisioning from other resources, and enables significant benefits through consolidation. We describe how DGSF solves GPU disaggregation challenges including supporting API transparency, hiding the latency of communication with remote GPUs, and load-balancing access to heavily shared GPUs. Evaluation of our prototype on six workloads shows that DGSF's API remoting optimizations can improve the runtime of a function by up to 50% relative to unoptimized DGSF. Such optimizations, which aggressively remove GPU runtime and object management latency from the critical path, can enable functions running over DGSF to have a lower end-to-end time than when running on a GPU natively. By enabling GPU sharing, DGSF can reduce function queueing latency by up to 53%. We use DGSF to augment AWS Lambda with GPU support, showing similar benefits.