A Lightweight Deep Recurrent Q-Learning Technique for Autonomous Wildfire Surveillance

We study the problem of wildfire surveillance using autonomous unmanned aerial vehicles (UAVs). The objective of the UAVs is to find the maximum number of locations that are under fire, assuming that the UAVs can share their observations. We propose a deep recurrent Q-learning technique that uses th...

Full description

Saved in:
Bibliographic Details
Main Authors: Cantor, Jeremy, Kreidl, Patrick, Nuszkowski, John, Harris, Alan, Dutta, Ayan
Format: Conference Proceeding
Language:English
Subjects:
Atmospheric modeling
Memory management
Neural networks
Q-learning
reinforcement learning
Surveillance
Training
wildfire surveillance
Wildfires
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We study the problem of wildfire surveillance using autonomous unmanned aerial vehicles (UAVs). The objective of the UAVs is to find the maximum number of locations that are under fire, assuming that the UAVs can share their observations. We propose a deep recurrent Q-learning technique that uses these observations to make decisions for the robots, i.e., where to move next. The prohibitively large state space underlying the decision policy motivates a neural network approximation, but prior work used only convolutional layers to extract spatial fire information from the current observations. Our network also incorporates a recurrent module to capture temporal information from the history of observations. Experiments involving two simulated fixed-wing aircraft feature a more realistic physics-based wildfire propagation model than the discrete wildfire models of prior work. Results show that our proposed technique uses about 20 times less memory than the approach of prior work, while performing comparably in terms of finding the fire's locations.
ISSN:1946-0759
DOI:10.1109/ICMLA58977.2023.00164