PowerTrain: Fast, generalizable time and power prediction models to optimize DNN training on accelerated edges

• Published in: Future generation computer systems 2024-12, Vol.161, p.329-344
• Main Authors: S.K., Prashanthi, Taluri, Saisamarth, S, Beautlin, Karwa, Lakshya, Simmhan, Yogesh
• Format: Article
• Language: English
• Subjects: DNN training; Edge accelerators; Edge computing; Performance modeling; Performance optimization
• ISSN: 0167-739X
• DOI: 10.1016/j.future.2024.07.001

Accelerated edge devices, like Nvidia’s Jetson with 1000+ CUDA cores, are increasingly used for DNN training and federated learning, rather than just for inferencing workloads. A unique feature of these compact devices is their fine-grained control over CPU, GPU, memory frequencies, and active CPU cores, which can limit their power envelope in a constrained setting while throttling the compute performance. Given this vast 10k+ parameter space, selecting a power mode for dynamically arriving training workloads to exploit power–performance trade-offs requires costly profiling for each new workload, or is done ad hoc. We propose PowerTrain, a transfer-learning approach to accurately predict the power and time that will be consumed when we train a given DNN workload (model + dataset) using any specified power mode (CPU/GPU/memory frequencies, core-count). It requires a one-time offline profiling of 1000s of power modes for a reference DNN workload on a single Jetson device (Orin AGX) to build Neural Network (NN) based prediction models for time and power. These NN models are subsequently transferred (retrained) for a new DNN workload, or even a different Jetson device, with minimal additional profiling of just 50 power modes to make accurate time and power predictions. These are then used to rapidly construct the Pareto front and select the optimal power mode for the new workload, e.g., to minimize training time while meeting a power limit. PowerTrain’s predictions are robust to new workloads, exhibiting a low MAPE of