Analysis validation has been neglected in the Age of Reproducibility

Increasingly complex statistical models are being used for the analysis of biological data. Recent commentary has focused on the ability to compute the same outcome for a given dataset (reproducibility). We argue that a reproducible statistical analysis is not necessarily valid because of unique pat...

Full description

Saved in:
Bibliographic Details
Published in:PLoS biology 2018-12, Vol.16 (12), p.e3000070-e3000070
Main Authors: Lotterhos, Kathleen E, Moore, Jason H, Stapleton, Ann E
Format: Article
Language:English
Subjects:
Biological activity
Biology
Biology and Life Sciences
Computational Biology - methods
Computational Biology - statistics & numerical data
Computer and Information Sciences
Computer simulation
Data Interpretation, Statistical
Datasets
Genomes
Grants
Historical account
Humans
Informatics
Mathematical models
Medical research
Medicine, Experimental
Methods
Models, Statistical
Nature
Physical Sciences
Population genetics
Principal components analysis
Reproducibility
Reproducibility of Results
Research and Analysis Methods
Statistical analysis
Statistical methods
Statistical models
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Increasingly complex statistical models are being used for the analysis of biological data. Recent commentary has focused on the ability to compute the same outcome for a given dataset (reproducibility). We argue that a reproducible statistical analysis is not necessarily valid because of unique patterns of nonindependence in every biological dataset. We advocate that analyses should be evaluated with known-truth simulations that capture biological reality, a process we call "analysis validation." We review the process of validation and suggest criteria that a validation project should meet. We find that different fields of science have historically failed to meet all criteria, and we suggest ways to implement meaningful validation in training and practice.
ISSN:1545-7885
1544-9173
1545-7885
DOI:10.1371/journal.pbio.3000070