Approaching the Symbol Grounding Problem with Probabilistic Graphical Models

In order for robots to engage in dialogue with human teammates, they must have the ability to identify correspondences between elements of language and aspects of the external world. A solution to this symbol‐grounding problem (Harnad, 1990) would enable a robot to interpret commands such as “Drive...

Full description

Saved in:
Bibliographic Details
Published in:The AI magazine 2011-12, Vol.32 (4), p.64-76
Main Authors: Tellex, Stefanie, Kollar, Thomas, Dickerson, Steven, Walter, Matthew R., Banerjee, Ashis Gopal, Teller, Seth, Roy, Nicholas
Format: Article
Language:English
Subjects:
Applied sciences
Artificial intelligence
Communication
Communications
Computational linguistics
Computer science
control theory
systems
Control theory. Systems
Exact sciences and technology
Human-machine systems
Information systems. Data bases
Language
Language processing
Learning and adaptive systems
Linguistics
Mathematical models
Memory organisation. Data processing
Methods
Natural language
Natural language interfaces
Natural language processing
Robot
Robotics
Robots
Semantics
Software
Speech and sound recognition and synthesis. Linguistics
Studies
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In order for robots to engage in dialogue with human teammates, they must have the ability to identify correspondences between elements of language and aspects of the external world. A solution to this symbol‐grounding problem (Harnad, 1990) would enable a robot to interpret commands such as “Drive over to receiving and pick up the tire pallet.” This article describes several of our results that use probabilistic inference to address the symbol‐grounding problem. Our approach is to develop models that factor according to the linguistic structure of a command. We first describe an early result, a generative model that factors according to the sequential structure of language, and then discuss our new framework, generalized grounding graphs (G3). The G3 framework dynamically instantiates a probabilistic graphical model for a natural language input, enabling a mapping between words in language and concrete objects, places, paths, and events in the external world. We report on corpus‐based experiments in which the robot is able to learn and use word meanings in three real‐world tasks: indoor navigation, spatial language video retrieval, and mobile manipulation.
ISSN:0738-4602
2371-9621
DOI:10.1609/aimag.v32i4.2384