Optimal bipartite consensus control for heterogeneous unknown multi-agent systems via reinforcement learning

• Published in: Applied mathematics and computation 2024-09, Vol.476, p.128785, Article 128785
• Main Authors: Meng, Hao, Pang, Denghao, Cao, Jinde, Guo, Yechen, Niazi, Azmat Ullah Khan
• Format: Article
• Language: English
• Subjects: Cooperative control; Heterogeneous multi-agent systems; Optimal bipartite consensus; Reinforcement learning
• ISSN: 0096-3003; 1873-5649
• DOI: 10.1016/j.amc.2024.128785

This study focuses on addressing optimal bipartite consensus control (OBCC) problems in heterogeneous multi-agent systems (MASs) without relying on the agents' dynamics. Motivated by the need for model-free and optimal consensus control in complex MASs, a novel distributed scheme utilizing reinforcement learning (RL) is proposed to overcome these challenges. The MAS network is randomly partitioned into sub-networks where agents collaborate within each subgroup to attain tracking control and ensure convergence of positions and speeds to a common value. However, agents from distinct subgroups compete to achieve diverse tracking objectives. Furthermore, the heterogeneous MASs considered have unknown first and second-order dynamics, adding to the complexity of the problem. To address the OBCC issue, the policy iteration (PI) algorithm is used to acquire solutions for discrete-time Hamilton-Jacobi-Bellman (HJB) equations while implementing a data-driven actor-critic neural network (ACNN) framework. Ultimately, the accuracy of our proposed approach is confirmed through the presentation of numerical simulations. •A novel approach is proposed to investigate the OBCC of heterogeneous MASs in competition-cooperation relationship.•To avoid the use of system dynamics, a model-free RL algorithm is proposed, utilizing available input-output data to construct a new system and achieve the OBCC of heterogeneous MASs.•The challenge of dimensional discrepancies in heterogeneous MASs is overcome by the incorporation of estimated velocities, thereby converting the heterogeneous systems into homogeneous systems.•A distributed actor-critic neural network based on PI is proposed to obtain the optimal control policy, and the convergence analysis of the PI algorithm is conducted.