EvalRS: a Rounded Evaluation of Recommender Systems

Much of the complexity of Recommender Systems (RSs) comes from the fact that they are used as part of more complex applications and affect user experience through a varied range of user interfaces. However, research focused almost exclusively on the ability of RSs to produce accurate item rankings w...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2022-08
Main Authors: Tagliabue, Jacopo, Bianchi, Federico, Schnabel, Tobias, Attanasio, Giuseppe, Greco, Ciro, Gabriel de Souza P Moreira, Chia, Patrick John
Format: Article
Language:English
Subjects:
Complexity
Recommender systems
Systems analysis
User experience
User interfaces
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Much of the complexity of Recommender Systems (RSs) comes from the fact that they are used as part of more complex applications and affect user experience through a varied range of user interfaces. However, research focused almost exclusively on the ability of RSs to produce accurate item rankings while giving little attention to the evaluation of RS behavior in real-world scenarios. Such narrow focus has limited the capacity of RSs to have a lasting impact in the real world and makes them vulnerable to undesired behavior, such as reinforcing data biases. We propose EvalRS as a new type of challenge, in order to foster this discussion among practitioners and build in the open new methodologies for testing RSs "in the wild".
ISSN:2331-8422