Metrics for Evaluating Social Conformity of Crowd Navigation Algorithms

Recent protocols and metrics for training and evaluating autonomous robot navigation through crowds are inconsistent due to diversified definitions of "social behavior". This makes it difficult, if not impossible, to effectively compare published navigation algorithms. Without a good evalu...

Full description

Saved in:
Bibliographic Details
Main Authors: Wang, Junxian, Chan, Wesley P., Carreno-Medrano, Pamela, Cosgun, Akansel, Croft, Elizabeth
Format: Conference Proceeding
Language:English
Subjects:
Complexity theory
Conferences
Measurement
Navigation
Protocols
Systematics
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recent protocols and metrics for training and evaluating autonomous robot navigation through crowds are inconsistent due to diversified definitions of "social behavior". This makes it difficult, if not impossible, to effectively compare published navigation algorithms. Without a good evaluation protocol resulting algorithms may fail to generalize, due to lack of diversity in training. To address these gaps, this paper facilitates a more comprehensive evaluation and objective comparison of crowd navigation algorithms by proposing a consistent set of metrics that accounts for both efficiency and social conformity. The paper also provides a systematic protocol comprising multiple crowd navigation scenarios of varying complexity for evaluation. We tested four state-of-the-art algorithms under this protocol. Results revealed that some state-of-the-art algorithms have great challenges in generalizing, and using our protocol for training, we were able to improve the algorithm's performance. We demonstrate that the set of proposed metrics provides more insight and effectively differentiates the performance of these algorithms with respect to efficiency and social conformity.
ISSN:2162-7576
DOI:10.1109/ARSO54254.2022.9802981