Output feedback fault-tolerant Q-learning for discrete-time linear systems with actuator faults

• Published in: Engineering applications of artificial intelligence 2024-12, Vol.138, p.109355, Article 109355
• Main Authors: Rafiee, Sajad, Kankashvar, Mohammadrasoul, Bolandi, Hossein
• Format: Article
• Language: English
• Subjects: Approximate dynamic programming; Fault-tolerant control; Linear quadratic regulator; Optimal control; Output feedback; Reinforcement learning
• ISSN: 0952-1976
• DOI: 10.1016/j.engappai.2024.109355

This paper presents an innovative output feedback fault-tolerant Q-learning algorithm that can be implemented online without relying on explicit system models or fault details. In the face of actuator faults, finding optimal Fault-Tolerant Control (FTC) solutions that can stabilize the faulty system poses significant challenges. The proposed approach is implemented online without the need for system dynamics and actuator fault information. Furthermore, it operates without relying on full state measurements, utilizing only the input-output data of the faulty system. An innovative representation of the output feedback Fault-Tolerant Q-function (FTQF) is established using input-output data. Subsequently, a model-free optimal output feedback FTC policy is obtained from the developed FTQF. Then, a fault-tolerant Q-learning algorithm is formulated to iteratively acquire the optimal FTC policy in real-time, eliminating the necessity for system dynamics and actuator fault information. The proposed algorithm exhibits immunity to excitation noise bias, even without considering a discounting factor. Furthermore, the proposed Q-learning approach is proven to be effective in stabilizing faulty closed-loop system. Finally, the proposed algorithm's efficiency is validated through numerical simulations of F-16 autopilot aircraft dynamics.