Automated variable selection methods for logistic regression produced unstable models for predicting acute myocardial infarction mortality

Automated variable selection methods are frequently used to determine the independent predictors of an outcome. The objective of this study was to determine the reproducibility of logistic regression models developed using automated variable selection methods. An initial set of 29 candidate variable...

Full description

Saved in:
Bibliographic Details
Published in:Journal of clinical epidemiology 2004-11, Vol.57 (11), p.1138-1146
Main Authors: Austin, Peter C., Tu, Jack V.
Format: Article
Language:English
Subjects:
Acute myocardial infarction
Aged
Automatic Data Processing
Epidemiology
Female
Heart attacks
Humans
Logistic Models
Logistic regression
Male
Medical research
Middle Aged
Mortality
Multivariate analysis
Myocardial infarction
Myocardial Infarction - mortality
Older people
Regression models
Reproducibility of Results
Selection Bias
Studies
Variable selection
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Automated variable selection methods are frequently used to determine the independent predictors of an outcome. The objective of this study was to determine the reproducibility of logistic regression models developed using automated variable selection methods. An initial set of 29 candidate variables were considered for predicting mortality after acute myocardial infarction (AMI). We drew 1,000 bootstrap samples from a dataset consisting of 4,911 patients admitted to hospital with an AMI. Using each bootstrap sample, logistic regression models predicting 30-day mortality were obtained using backward elimination, forward selection, and stepwise selection. The agreement between the different model selection methods and the agreement across the 1,000 bootstrap samples were compared. Using 1,000 bootstrap samples, backward elimination identified 940 unique models for predicting mortality. Similar results were obtained for forward and stepwise selection. Three variables were identified as independent predictors of mortality among all bootstrap samples. Over half the candidate prognostic variables were identified as independent predictors in less than half of the bootstrap samples. Automated variable selection methods result in models that are unstable and not reproducible. The variables selected as independent predictors are sensitive to random fluctuations in the data.
ISSN:0895-4356
1878-5921
DOI:10.1016/j.jclinepi.2004.04.003