GEM-RL: Generalized Energy Management of Wearable Devices using Reinforcement Learning

• Main Authors: Basaklar, Toygun, Tuncel, Yigit, Gumussoy, Suat, Ogras, Umit
• Format: Conference Proceeding
• Language: English
• Subjects: Batteries; dynamic programming; Energy consumption; Energy harvesting; energy management; Hardware; multi-objective reinforcement learning; Reinforcement learning; Time measurement; Uncertainty; Wearable computers
• ISSN: 1558-1101
• DOI: 10.23919/DATE56975.2023.10137228

Energy harvesting (EH) and management (EM) have emerged as enablers of self-sustained wearable devices. Since EH alone is not sufficient for self-sustainability due to uncertainties of ambient sources and user activities, there is a critical need for a user-independent EM approach that does not rely on expected EH predictions. We present a generalized energy management framework (GEM-RL) using multi-objective reinforcement learning. GEM-RL learns the trade-off between utilization and the battery energy level of the target device under dynamic EH patterns and battery conditions. It also uses a lightweight approximate dynamic programming (ADP) technique that utilizes the trained MORL agent to optimize the utilization of the device over a longer period. Thorough experiments show that, on average, GEM-RL achieves Pareto front solutions within 5.4% of the offline Oracle for a given day. For a 7-day horizon, it achieves utility up to 4% within the offline Oracle and up to 50% higher utility compared to baseline EM approaches. The hardware implementation on a wearable device shows negligible execution time (1.98 ms) and energy consumption (23.17 μJ) overhead.