Multi-Objective Deep Reinforcement Learning for Crowd Route Guidance Optimization

• Published in: Transportation research record 2024-05, Vol.2678 (5), p.617-633
• Main Authors: Nishida, Ryo, Tanigaki, Yuki, Onishi, Masaki, Hashimoto, Koichi
• Format: Article
• Language: English
• ISSN: 0361-1981; 2169-4052
• DOI: 10.1177/03611981231190635

In this study, we propose an improved version of Pareto deep Q-network (PDQN), a multi-objective deep reinforcement learning method, and attempt to demonstrate its effectiveness in a real-world problem such as crowd route guidance strategy optimization. Overcrowding during crowd movement can sometimes lead to accidents; therefore, it is imperative to guide crowds to move safely and efficiently. Safety and efficiency are conflicting objectives, and how to dynamically determine guidance can be formulated as a multi-objective sequential decision-making problem. PDQN is suitable for solving these problems, but its applicability to complex real-world problems has not been fully verified. To apply PDQN to real problems, we propose to adjust the parameters of PDQN and improve the action selection criteria during learning. A toy problem and a crowd guidance problem using multi-agent crowd simulation are adopted to evaluate the performance of improved PDQN. Experimental results show that the improved PDQN can search for good strategies compared with the original PDQN and can obtain better strategies than simplified strategies.