FaaSCtrl: A Comprehensive-Latency Controller for Serverless Platforms

• Published in: IEEE transactions on cloud computing 2024-10, Vol.12 (4), p.1328-1343
• Main Authors: Panda, Abhisek, Sarangi, Smruti R.
• Format: Article
• Language: English
• Subjects: Complexity; Degradation; Design optimization; Design standards; Distributed processing; Job colocation; Knobs; Measurement; Optimization; performance interference; Performance measurement; Priority scheduling; Quality of service; Real time; Real-time systems; Reinforcement learning; resource scheduling; Schedules; Serverless computing; Tail
• ISSN: 2168-7161; 2372-0018
• DOI: 10.1109/TCC.2024.3473015

Serverless computing systems have become very popular because of their natural advantages with respect to auto-scaling, load balancing and fast distributed processing. As of today, almost all serverless systems define two QoS classes: best-effort (BE BE ) and latency-sensitive (LS LS ). Systems typically do not offer any latency or QoS guarantees for BE BE jobs and run them on a best-effort basis. In contrast, systems strive to minimize the processing time for LS LS jobs. This work proposes a precise definition for these job classes and argues that we need to consider a bouquet of performance metrics for serverless applications, not just a single one. We thus propose the comprehensive latency (CL CL ) that comprises the mean, tail latency, median and standard deviation of a series of invocations for a given serverless function. Next, we design a system FaaSCtrl , whose main objective is to ensure that every component of the CL CL is within a prespecified limit for an LS application, and for BE applications, these components are minimized on a best-effort basis. Given the sheer complexity of the scheduling problem in a large multi-application setup, we use the method of surrogate functions in optimization theory to design a simpler optimization problem that relies on performanc