Improvement of Risk Prediction by Genomic Profiling: Reclassification Measures Versus the Area Under the Receiver Operating Characteristic Curve

• Published in: American journal of epidemiology 2010-08, Vol.172 (3), p.353-361
• Main Authors: Mihaescu, Raluca, van Zitteren, Moniek, van Hoek, Mandy, Sijbrands, Eric J. G., Uitterlinden, André G., Witteman, Jacqueline C. M., Hofman, Albert, Hunink, M. G. Myriam, van Duijn, Cornelia M., Janssens, A. Cecile J. W.
• Format: Article
• Language: English
• Subjects: Biological and medical sciences; Diabetes; diabetes mellitus; Diabetes. Impaired glucose tolerance; DNA Fingerprinting - statistics & numerical data; Endocrine pancreas. Apud cells (diseases); Endocrinopathies; Epidemiology; Etiopathogenesis. Screening. Investigations. Target tissue resistance; General aspects; genetic predisposition to disease; Genetics; Humans; Medical sciences; Metagenomics - statistics & numerical data; Miscellaneous; Polymorphism; Prospective Studies; Public health. Hygiene; Public health. Hygiene-occupational medicine; Risk assessment; Risk Factors; ROC Curve; single nucleotide; statistical; type 2
• ISSN: 0002-9262; 1476-6256
• DOI: 10.1093/aje/kwq122

Reclassification is observed even when there is no or minimal improvement in the area under the receiver operating characteristic curve (AUC), and it is unclear whether it indicates improved clinical utility. The authors investigated total reclassification, net reclassification improvement, and integrated discrimination improvement for different ΔAUC using empirical and simulated data. Empirical analyses compared prediction of type 2 diabetes risk based on age, sex, and body mass index with prediction updated with 18 established genetic risk factors. Simulated data were used to investigate measures of reclassification against ΔAUCs of 0.005, 0.05, and 0.10. Total reclassification and net reclassification improvement were calculated for all possible cutoff values. The AUC of type 2 diabetes risk prediction improved from 0.63 to 0.66 when 18 polymorphisms were added, whereas total reclassification ranged from 0% to 22.5% depending on the cutoff value chosen. In the simulation study, total reclassification, net reclassification improvement, and integrated discrimination improvement increased with higher ΔAUC. When ΔAUC was low (0.005), net reclassification improvement values were close to zero, integrated discrimination improvement was 0.08% (P > 0.05), but total reclassification ranged from 0 to 6.7%. Reclassification increases with increasing AUC but predominantly varies with the cutoff values chosen. Reclassification observed in the absence of AUC increase is unlikely to improve clinical utility.