Cognitive Mapping and Planning for Visual Navigation

We introduce a neural architecture for navigation in novel environments. Our proposed architecture learns to map from first-person views and plans a sequence of actions towards goals in the environment. The Cognitive Mapper and Planner (CMP) is based on two key ideas: (a) a unified joint architectur...

Full description

Saved in:
Bibliographic Details
Published in:International journal of computer vision 2020-05, Vol.128 (5), p.1311-1330
Main Authors: Gupta, Saurabh, Tolani, Varun, Davidson, James, Levine, Sergey, Sukthankar, Rahul, Malik, Jitendra
Format: Article
Language:English
Subjects:
Architecture
Artificial Intelligence
Computer Imaging
Computer Science
Computer simulation
Image Processing and Computer Vision
Indoor environments
Mapping
Memory tasks
Navigation
Path planning
Pattern Recognition
Pattern Recognition and Graphics
Planning
Spatial analysis
Special Issue on Deep Learning for Robotic Vision
Vision
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We introduce a neural architecture for navigation in novel environments. Our proposed architecture learns to map from first-person views and plans a sequence of actions towards goals in the environment. The Cognitive Mapper and Planner (CMP) is based on two key ideas: (a) a unified joint architecture for mapping and planning, such that the mapping is driven by the needs of the task, and (b) a spatial memory with the ability to plan given an incomplete set of observations about the world. CMP constructs a top-down belief map of the world and applies a differentiable neural net planner to produce the next action at each time step. The accumulated belief of the world enables the agent to track visited regions of the environment. We train and test CMP on navigation problems in simulation environments derived from scans of real world buildings. Our experiments demonstrate that CMP outperforms alternate learning-based architectures, as well as, classical mapping and path planning approaches in many cases. Furthermore, it naturally extends to semantically specified goals, such as “going to a chair”. We also deploy CMP on physical robots in indoor environments, where it achieves reasonable performance, even though it is trained entirely in simulation.
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-019-01236-7