Image-Based Multi-Agent Reinforcement Learning for Demand–Capacity Balancing

Air traffic flow management (ATFM) is of crucial importance to the European Air Traffic Control System due to two factors: first, the impact of ATFM, including safety implications on ATC operations; second, the possible consequences of ATFM measures on both airports and airlines operations. Thus, th...

Full description

Saved in:
Bibliographic Details
Published in:Aerospace 2022-10, Vol.9 (10), p.599
Main Authors: Mas-Pujol, Sergi, Salamí, Esther, Pastor, Enric
Format: Article
Language:English
Subjects:
Air traffic control
air traffic flow management
Air traffic management
Airline operations
Airlines
Airports
Airspace
Algorithms
Artificial intelligence
Aviation
Balancing
Collaboration
Control systems
deep deterministic policy gradient
deep Q-learning
Demand
demand–capacity balancing
Integer programming
Machine learning
multi-agent
Multiagent systems
Optimization techniques
reinforcement learning
Supply and demand
Traffic
Traffic capacity
Traffic congestion
Traffic flow
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Air traffic flow management (ATFM) is of crucial importance to the European Air Traffic Control System due to two factors: first, the impact of ATFM, including safety implications on ATC operations; second, the possible consequences of ATFM measures on both airports and airlines operations. Thus, the central flow management unit continually seeks to improve traffic flow management to reduce delays and congestion. In this work, we investigated the use of reinforcement learning (RL) methods to compute policies to solve demand–capacity imbalances (a.k.a. congestion) during the pre-tactical phase. To address cases where the expected demands exceed the airspace sector capacity, we considered agents representing flights who have to decide on ground delays jointly. To overcome scalability issues, we propose using raw pixel images as input, which can represent an arbitrary number of agents without changing the system’s architecture. This article compares deep Q-learning and deep deterministic policy gradient algorithms with different configurations. Experimental results, using real-world data for training and validation, confirm the effectiveness of our approach to resolving demand–capacity balancing problems, showing the robustness of the RL approach presented in this article.
ISSN:2226-4310
2226-4310
DOI:10.3390/aerospace9100599