Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning

In recent years, multi-agent reinforcement learning algorithms have made significant advancements in diverse gaming environments, leading to increased interest in the broader application of such techniques. To address the prevalent challenge of partial observability, communication-based algorithms h...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-04
Main Authors: Li, Dapeng, Dong, Hang, Wang, Lu, Qiao, Bo, Qin, Si, Lin, Qingwei, Zhang, Dongmei, Zhang, Qi, Xu, Zhiwei, Zhang, Bin, Fan, Guoliang
Format: Article
Language:English
Subjects:
Algorithms
Large language models
Machine learning
Messages
Modules
Multiagent systems
Verbal communication
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In recent years, multi-agent reinforcement learning algorithms have made significant advancements in diverse gaming environments, leading to increased interest in the broader application of such techniques. To address the prevalent challenge of partial observability, communication-based algorithms have improved cooperative performance through the sharing of numerical embedding between agents. However, the understanding of the formation of collaborative mechanisms is still very limited, making designing a human-understandable communication mechanism a valuable problem to address. In this paper, we propose a novel multi-agent reinforcement learning algorithm that embeds large language models into agents, endowing them with the ability to generate human-understandable verbal communication. The entire framework has a message module and an action module. The message module is responsible for generating and sending verbal messages to other agents, effectively enhancing information sharing among agents. To further enhance the message module, we employ a teacher model to generate message labels from the global view and update the student model through Supervised Fine-Tuning (SFT). The action module receives messages from other agents and selects actions based on current local observations and received messages. Experiments conducted on the Overcooked game demonstrate our method significantly enhances the learning efficiency and performance of existing methods, while also providing an interpretable tool for humans to understand the process of multi-agent cooperation.
ISSN:2331-8422