Fractional-order Q-learning based on modal decomposition and convolutional neural networks for voltage control of smart grids

• Published in: Applied soft computing 2024-09, Vol.162, p.111825, Article 111825
• Main Authors: Yin, Linfei, Mo, Nan
• Format: Article
• Language: English
• Subjects: Complementary ensemble empirical mode decomposition with adaptive noise; Convolutional neural networks; Fractional-order proportional-integral-derivative; Q-learning; Voltage control
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2024.111825

To maintain uniformity in the time scale of the control system, a coordinated first-level voltage control (FVC) framework is proposed for the voltage controller of three-state energy (TSE) models. To ensure control accuracy, this study combines complementary ensemble empirical mode decomposition (CEEMD) with adaptive noise (CEEMDAN), fractional-order proportional-integral-derivative (FO-PID) with convolutional neural networks (CNNs), Q-learning as CFOQL. Firstly, the CEEMDAN decomposes the historical voltage deviation data. The decomposed intrinsic mode functions (IMFs) are converted into RGB images with 30×30 pixels to train the CNNs. The residual function (RF) is utilized to train Q-learning. The implemented IMFs on the trained CNNs output, can generate regulating commands to adjust the differential and integral coefficients of FO-PID control. Similarly, the implemented RF on the trained Q-learning output can generate regulating commands. Then, the trained CNNs output the differential and integral coefficients for FO-PID control; Q-learning generates the regulating commands based on the RF. Finally, the regulation commands generated by Q-learning are combined with the regulation commands generated by the FO-PID as the total regulation command. The proposed controller can effectively reduce the voltage deviations of smart grids (SGs) with higher control accuracy. Significantly, in the IEEE 2736-bus system, the error integration criterion indices obtained by the proposed controller are at least 7.53 %, 1.89 %, 13.80 %, and 14.61 % less than those of the proportional-integral-derivative (PID) algorithm, FO-PID algorithm, R(λ) algorithm, and Q(λ) algorithm, respectively. •A coordinated first-level voltage control framework with unified time scale is built.•Reactive power regulation problem by collecting the historical voltage is considered.•Empirical modal decomposition is embedded into a real-time voltage controller.•Parameters of the designed controller are predicted by convolutional neural networks.•Fractional-order proportional-integral-derivative and Q-learning are combined.