Sample size estimation using the receiver operating characteristic curve

In This work we describe two related approaches to estimating the sample sizes required to statistically compare the performance of two classifiers: acceptable failure rates (AFR) and the area under the receiver operating characteristic (ROC) curve (AUC). In particular, we consider rare event detect...

Full description

Saved in:
Bibliographic Details
Main Authors: Bradley, A.P., Longstaff, I.D.
Format: Conference Proceeding
Language:English
Subjects:
Australia
Event detection
Information processing
Information technology
Particle measurements
Pattern recognition
Probability
Sensor phenomena and characterization
Signal processing
Testing
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In This work we describe two related approaches to estimating the sample sizes required to statistically compare the performance of two classifiers: acceptable failure rates (AFR) and the area under the receiver operating characteristic (ROC) curve (AUC). In particular, we consider rare event detection problems, where the prior class probabilities are highly skewed, and measure performance at a specific operating point and for the whole ROC curve. It is shown that the use of AUC as a performance measure is preferable to AFR as it requires a smaller data set to demonstrate superiority of one classifier over another.
ISSN:1051-4651
2831-7475
DOI:10.1109/ICPR.2004.1333794