Reinforcement Learning-Based Control of DC-DC Buck Converter Considering Controller Time Delay

• Published in: IEEE access 2024, Vol.12, p.118442-118452
• Main Authors: Lee, Donghun, Kim, Bongseok, Kwon, Soonhyung, Nguyen, Ngoc-Duc, Kyu Sim, Min, Il Lee, Young
• Format: Article
• Language: English
• Subjects: Buck converters; DC-DC power converters; DC-DC synchronous buck converter; Delay effects; Delays; Digital signal processing; digital signal processor (DSP); Inductors; optimal control; Pulse width modulation; real-time deep reinforcement learning (RTDRL); Real-time systems; Reinforcement learning; Tuning
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3448535

Non-linearities and unmodeled dynamics in the control system inevitably degrade the quality and reliability of voltage stabilization performance in DC-DC buck converters. Reinforcement Learning (RL) is an emerging method to mitigate this issue. However, traditional RL typically necessitates significant computational resources and specialized processing units, thus being an economically unreasonable option. This paper proposes a high-performance RL-based method even suitable for a cost-effective Digital Signal Processor (DSP). To address the significant challenge of time delay in a DSP when training the RL agent, this paper adopts a Real-Time Deep Reinforcement Learning (RTDRL) approach that creates an augmented virtual decision process to eliminate the delay effect. The performance is validated through software simulation (PLECS) and an actual system, through which the proposed approach demonstrated superior performance compared to existing benchmarks, including existing approaches and artificial intelligence.