A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette methods for receiver operating characteristic (ROC) data

There are several different statistical methods for analysing multireader ROC studies, with the Dorfman–Berbaum–Metz (DBM) method being the most frequently used. Another method is the corrected F method proposed by Obuchowski and Rockette (OR). The DBM and OR procedures at first appear quite differe...

Full description

Saved in:
Bibliographic Details
Published in:Statistics in medicine 2005-05, Vol.24 (10), p.1579-1607
Main Authors: Hillis, Stephen L., Obuchowski, Nancy A., Schartz, Kevin M., Berbaum, Kevin S.
Format: Article
Language:English
Subjects:
Analysis of Variance
Comparative analysis
corrected F
DBM
diagnostic radiology
jackknife
Medical research
Methods
pseudovalues
Radiography - statistics & numerical data
ROC Curve
Software packages
Statistical analysis
United States
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:There are several different statistical methods for analysing multireader ROC studies, with the Dorfman–Berbaum–Metz (DBM) method being the most frequently used. Another method is the corrected F method proposed by Obuchowski and Rockette (OR). The DBM and OR procedures at first appear quite different: DBM is a three‐way ANOVA analysis of pseudovalues while OR is a two‐way ANOVA analysis of accuracy estimates with correlated errors. We show that the original DBM and OR F statistics for testing the null hypothesis of equal treatments have the same form and will typically have similar values; however, differences in the denominator degrees of freedom will result in differences in p‐values even when the F statistics are identical. We show how the methods can be generalized to include variations in the accuracy measure, covariance method, and degrees of freedom. Identical results are obtained when the methods agree with respect to all three of these procedure parameters; hence for a particular choice of procedure parameters the choice of method appears to depend mainly on software preference and availability. The methods are compared using data from a factorial study with two modalities, five readers, and 114 patients. Copyright © 2005 John Wiley & Sons, Ltd.
ISSN:0277-6715
1097-0258
DOI:10.1002/sim.2024