Sample sizes of prediction model studies in prostate cancer were rarely justified and often insufficient

Developing clinical prediction models (CPMs) on data of sufficient sample size is critical to help minimize overfitting. Using prostate cancer as a clinical exemplar, we aimed to investigate to what extent existing CPMs adhere to recent formal sample size criteria, or historic rules of thumb of even...

Full description

Saved in:
Bibliographic Details
Published in:Journal of clinical epidemiology 2021-05, Vol.133, p.53-60
Main Authors: Collins, Shane D., Peek, Niels, Riley, Richard D., Martin, Glen P.
Format: Article
Language:English
Subjects:
Criteria
Design parameters
Development
Epidemiology
Medical prognosis
Parameter identification
Prediction models
Prostate cancer
Sample size
Systematic review
Validation
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Developing clinical prediction models (CPMs) on data of sufficient sample size is critical to help minimize overfitting. Using prostate cancer as a clinical exemplar, we aimed to investigate to what extent existing CPMs adhere to recent formal sample size criteria, or historic rules of thumb of events per predictor parameter (EPP)≥10. A systematic review to identify CPMs related to prostate cancer, which provided enough information to calculate minimum sample size. We compared the reported sample size of each CPM against the traditional 10 EPP rule of thumb and formal sample size criteria. About 211 CPMs were included. Three of the studies justified the sample size used, mostly using EPP rules of thumb. Overall, 69% of the CPMs were derived on sample sizes that surpassed the traditional EPP≥10 rule of thumb, but only 48% surpassed recent formal sample size criteria. For most CPMs, the required sample size based on formal criteria was higher than the sample sizes to surpass 10 EPP. Few of the CPMs included in this study justified their sample size, with most justifications being based on EPP. This study shows that, in real-world data sets, adhering to the classic EPP rules of thumb is insufficient to adhere to recent formal sample size criteria. •Sample Size justification of 211 CPMs related to prostate cancer was lacking.•An EPP≥10 is not necessarily sufficient to guide sample size for CPM development.•Sample sizes for CPM development should follow formal sample size criteria.•We recommend sample size justification in all future prediction model studies.
ISSN:0895-4356
1878-5921
DOI:10.1016/j.jclinepi.2020.12.011