Benchmarking parallelism in FaaS platforms

• Published in: Future generation computer systems 2021-11, Vol.124, p.268-284
• Main Authors: Barcelona-Pons, Daniel, García-López, Pedro
• Format: Article
• Language: English
• Subjects: Benchmark; FaaS; Parallelism; Serverless
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2021.06.005

Serverless computing has seen a myriad of work exploring its potential. Some systems tackle Function-as-a-Service (FaaS) properties on automatic elasticity and scale to run highly-parallel computing jobs. However, they focus on specific platforms and convey that their ideas can be extrapolated to any FaaS runtime. An important question arises: do all FaaS platforms fit parallel computations? In this paper, we argue that not all of them provide the necessary means to host highly-parallel applications. To validate our hypothesis, we create a comparative framework and categorize the architectures of four cloud FaaS offerings, emphasizing parallel performance. We attest and extend this description with an empirical experiment that consists in plotting in deep detail the evolution of a parallel computing job on each service. The analysis of our results evinces that FaaS is not inherently good for parallel computations and architectural differences across platforms are decisive to categorize their performance. A key insight is the importance of virtualization technologies and the scheduling approach of FaaS platforms. Parallelism improves with lighter virtualization and proactive scheduling due to finer resource allocation and faster elasticity. This causes some platforms like AWS and IBM to perform well for highly-parallel computations, while others such as Azure present difficulties to achieve the required parallelism degree. Consequently, the information in this paper becomes of special interest to help users choose the most adequate infrastructure for their parallel applications. •FaaS is not inherently good for parallel computations, depending on implementation.•Parallel performance changes noticeably between Cloud FaaS platforms.•AWS and IBM support parallelism; Azure and in less degree Google present problems.•Parallelism improves with light virtualization and push-based architectures.