Emergent behaviors and scalability for multi-agent reinforcement learning-based pedestrian models

• Published in: Simulation modelling practice and theory 2017-05, Vol.74, p.117-133
• Main Authors: Martinez-Gil, Francisco, Lozano, Miguel, Fernández, Fernando
• Format: Article
• Language: English
• Subjects: Behavioural simulation; Emergent behaviours; Multi-Agent Reinforcement Learning (MARL); Pedestrian simulation and modeling
• ISSN: 1569-190X; 1878-1462
• DOI: 10.1016/j.simpat.2017.03.003

This paper analyzes the emergent behaviors of pedestrian groups that learn through the multiagent reinforcement learning model developed in our group. Five scenarios studied in the pedestrian model literature, and with different levels of complexity, were simulated in order to analyze the robustness and the scalability of the model. Firstly, a reduced group of agents must learn by interaction with the environment in each scenario. In this phase, each agent learns its own kinematic controller, that will drive it at a simulation time. Secondly, the number of simulated agents is increased, in each scenario where agents have previously learnt, to test the appearance of emergent macroscopic behaviors without additional learning. This strategy allows us to evaluate the robustness and the consistency and quality of the learned behaviors. For this purpose several tools from pedestrian dynamics, such as fundamental diagrams and density maps, are used. The results reveal that the developed model is capable of simulating human-like micro and macro pedestrian behaviors for the simulation scenarios studied, including those where the number of pedestrians has been scaled by one order of magnitude with respect to the situation learned.