The role of the p-value in the multitesting problem

Modern science frequently involves the analysis of large amount of quantitative information and the simultaneous testing of thousands or even hundreds of thousands null hypotheses. In this context, sometimes, naive deductions derived from the statistical reports substitute the rational thinking. The...

Full description

Saved in:
Bibliographic Details
Published in:Journal of applied statistics 2020-07, Vol.47 (9), p.1529-1542
Main Authors: Martínez-Camblor, P., Pérez-Fernández, S., Díaz-Coto, S.
Format: Article
Language:English
Subjects:
(Bio)markers
Context
Decision making
Diagnostic systems
false-discovery rate
Hypotheses
Hypothesis testing
multitesting problem
Null hypothesis
p-value
receiver-operating characteristic (ROC) curve
Statistical analysis
Statistical methods
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Modern science frequently involves the analysis of large amount of quantitative information and the simultaneous testing of thousands or even hundreds of thousands null hypotheses. In this context, sometimes, naive deductions derived from the statistical reports substitute the rational thinking. The reproducibility crisis is a direct consequence of the misleading statistical conclusions. In this paper, the authors revisit some of the controversies on the implications derived from the statistical hypothesis testing. They focus on the role of the p-value on the massive multitesting problem and the loss of its standard probabilistic interpretation. The analogy between the hypothesis tests and the usual diagnostic process (both involve a decision-making) is used to point out some limitations in the probabilistic p-value interpretation and to introduce the receiver-operating characteristic, ROC, curve as a useful tool in the large-scale multitesting context. The analysis of the well-known Hedenfalk data illustrates the problem.
ISSN:0266-4763
1360-0532
DOI:10.1080/02664763.2019.1682128