Comparison of rank aggregation methods based on inherent ability

Ranking is a common task for selecting and evaluating alternatives. In the past few decades, combining rankings results from various sources into a consensus ranking has become an increasingly active research topic. In this study, we focus on the evaluation of rank aggregation methods. We first deve...

Full description

Saved in:
Bibliographic Details
Published in:Naval research logistics 2017-10, Vol.64 (7), p.556-565
Main Authors: Xiao, Yu, Deng, Ye, Wu, Jun, Deng, Hong‐Zhong, Lu, Xin
Format: Article
Language:English
Subjects:
Accuracy
Aggregation
comparison
decision analysis
Evaluation
Experimental data
experimental data generation method
Ground truth
inherent ability
Methods
Pricing policies
Production planning
rank aggregation
Ranking
Ratings & rankings
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Ranking is a common task for selecting and evaluating alternatives. In the past few decades, combining rankings results from various sources into a consensus ranking has become an increasingly active research topic. In this study, we focus on the evaluation of rank aggregation methods. We first develop an experimental data generation method, which can provide ground truth ranking for alternatives based on their “inherent ability.” This experimental data generation method can generate the required individual synthetic rankings with adjustable accuracy and length. We propose characterizing the effectiveness of rank aggregation methods by calculating the Kendall tau distance between the aggregated ranking and the ground truth ranking. We then compare four classical rank aggregation methods and present some useful findings on the relative performances of the four methods. The results reveal that both the accuracy and length of individual rankings have a remarkable effect on the comparison results between rank aggregation methods. Our methods and results may be helpful to both researchers and decision‐makers.
ISSN:0894-069X
1520-6750
DOI:10.1002/nav.21771