Position Paper: Rethinking Empirical Research in Machine Learning: Addressing Epistemic and Methodological Challenges of Experimentation

We warn against a common but incomplete understanding of empirical research in machine learning (ML) that leads to non-replicable results, makes findings unreliable, and threatens to undermine progress in the field. To overcome this alarming situation, we call for more awareness of the plurality of...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-05
Main Authors: Herrmann, Moritz, Lange, F Julian D, Eggensperger, Katharina, Casalicchio, Giuseppe, Wever, Marcel, Feurer, Matthias, Rügamer, David, Hüllermeier, Eyke, Boulesteix, Anne-Laure, Bischl, Bernd
Format: Article
Language:English
Subjects:
Machine learning
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We warn against a common but incomplete understanding of empirical research in machine learning (ML) that leads to non-replicable results, makes findings unreliable, and threatens to undermine progress in the field. To overcome this alarming situation, we call for more awareness of the plurality of ways of gaining knowledge experimentally but also of some epistemic limitations. In particular, we argue most current empirical ML research is fashioned as confirmatory research while it should rather be considered exploratory.
ISSN:2331-8422