Performance evaluation of autonomous mobile robots

The National Institute of Standards and Technology (NIST) has initiated a program to develop quantitative metrics for machine intelligence. One of the possible approaches to evaluating machine intelligence is task-based performance testing, like a mouse in a maze. A series of application-specific te...

Full description

Saved in:
Bibliographic Details
Published in:Industrial robot 2002-06, Vol.29 (3), p.259-267
Main Authors: Jacoff, Adam, Messina, Elena, Evans, John
Format: Article
Language:English
Subjects:
Algorithms
Artificial intelligence
Behavior
Building failures
Business metrics
Design
Evacuations & rescues
Evaluation
Knowledge representation
Localization
Perception
Performance evaluation
Researchers
Robotics
Robots
Sensors
Standardized tests
Studies
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The National Institute of Standards and Technology (NIST) has initiated a program to develop quantitative metrics for machine intelligence. One of the possible approaches to evaluating machine intelligence is task-based performance testing, like a mouse in a maze. A series of application-specific testbeds is envisioned. NIST has created a set of reference test arenas for evaluating the performance of autonomous mobile robots performing urban search and rescue tasks. Robots must explore the maze-like test course, map the environment, find the simulated victims, and then report back their findings. In this paper, we describe our approach toward developing performance metrics for autonomous mobile robots through standardized testing within representative environments and objective performance evaluations. Our intent is to help accelerate the robotic research communities' advancement of mobile robot capabilities, thereby improving the effectiveness of robots performing within industrial settings, hazardous environments, and in exploration.
ISSN:0143-991X
1758-5791
DOI:10.1108/01439910210425568