Fine-Grained Heterogeneous Execution Framework with Energy Aware Scheduling

The growing convergence of high-performance, data analytics, and machine-learning applications is increasingly pushing computing systems toward heterogeneous processors and specialized hardware accelerators. Hardware heterogeneity, in turn, leads to finer-grained workflows. State-of-the-art server-l...

Full description

Saved in:
Bibliographic Details
Main Authors: Rattihalli, Gourav, Hogade, Ninad, Dhakal, Aditya, Frachtenberg, Eitan, Hong Enriquez, Rolando Pablo, Bruel, Pedro, Mishra, Alok, Milojicic, Dejan
Format: Conference Proceeding
Language:English
Subjects:
Cloud computing
Data analysis
Energy consumption
Energy-awareness
Fine-granularity
Framework
Graphics processing units
Heterogeneity
Machine learning
Processor scheduling
Scheduler
Schedules
Serverless
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The growing convergence of high-performance, data analytics, and machine-learning applications is increasingly pushing computing systems toward heterogeneous processors and specialized hardware accelerators. Hardware heterogeneity, in turn, leads to finer-grained workflows. State-of-the-art server-less computing resource managers do not currently provide efficient scheduling of such fine-grained tasks on systems with heterogeneous CPUs and specialized hardware accelerators (e.g., GPUs and FPGAs). Working with fine-grained tasks presents an opportunity for more efficient energy use via new scheduling models. Our proposed scheduler enables technologies like Nvidia's Multi-Process Service (MPS) to pack multiple fine-grained tasks on GPUs efficiently. Its advantages include better co-location of jobs and better sharing of hardware resources such as GPUs that were not previously possible on container orchestration systems. We propose a Kubernetes-native energy-aware scheduler that integrates with our heterogeneous framework. Combining fine-grained resource scheduling on heterogeneous hardware and energy-aware scheduling results in up to 17.6% improvement in makespan, up to 20.16% reduction in energy consumption for CPU workloads, and up to 58.15% improvement in makespan, and up to 28.92% reduction in energy consumption for GPU workloads.
ISSN:2159-6190
DOI:10.1109/CLOUD60044.2023.00014