Sample sizes of prediction model studies in prostate cancer were rarely justified and often insufficient

• Published in: Journal of clinical epidemiology 2021-05, Vol.133, p.53-60
• Main Authors: Collins, Shane D., Peek, Niels, Riley, Richard D., Martin, Glen P.
• Format: Article
• Language: English
• Subjects: Criteria; Design parameters; Development; Epidemiology; Medical prognosis; Parameter identification; Prediction models; Prostate cancer; Sample size; Systematic review; Validation
• ISSN: 0895-4356; 1878-5921
• DOI: 10.1016/j.jclinepi.2020.12.011

Developing clinical prediction models (CPMs) on data of sufficient sample size is critical to help minimize overfitting. Using prostate cancer as a clinical exemplar, we aimed to investigate to what extent existing CPMs adhere to recent formal sample size criteria, or historic rules of thumb of events per predictor parameter (EPP)≥10. A systematic review to identify CPMs related to prostate cancer, which provided enough information to calculate minimum sample size. We compared the reported sample size of each CPM against the traditional 10 EPP rule of thumb and formal sample size criteria. About 211 CPMs were included. Three of the studies justified the sample size used, mostly using EPP rules of thumb. Overall, 69% of the CPMs were derived on sample sizes that surpassed the traditional EPP≥10 rule of thumb, but only 48% surpassed recent formal sample size criteria. For most CPMs, the required sample size based on formal criteria was higher than the sample sizes to surpass 10 EPP. Few of the CPMs included in this study justified their sample size, with most justifications being based on EPP. This study shows that, in real-world data sets, adhering to the classic EPP rules of thumb is insufficient to adhere to recent formal sample size criteria. •Sample Size justification of 211 CPMs related to prostate cancer was lacking.•An EPP≥10 is not necessarily sufficient to guide sample size for CPM development.•Sample sizes for CPM development should follow formal sample size criteria.•We recommend sample size justification in all future prediction model studies.