Overinterpretation of findings in machine learning prediction model studies in oncology: a systematic review

In biomedical research, spin is the overinterpretation of findings, and it is a growing concern. To date, the presence of spin has not been evaluated in prognostic model research in oncology, including studies developing and validating models for individualized risk prediction. We conducted a system...

Full description

Saved in:
Bibliographic Details
Published in:Journal of clinical epidemiology 2023-05, Vol.157, p.120-133
Main Authors: Dhiman, Paula, Ma, Jie, Andaur Navarro, Constanza L., Speich, Benjamin, Bullock, Garrett, Damen, Johanna A.A., Hooft, Lotty, Kirtley, Shona, Riley, Richard D., Van Calster, Ben, Moons, Karel G.M., Collins, Gary S.
Format: Article
Language:English
Subjects:
Artificial intelligence
Cancer
Clinical medicine
Epidemiology
Humans
Learning algorithms
Machine Learning
Medical Oncology
Medical prognosis
Medical research
Neural networks
Oncology
Prediction model
Prediction models
Prognosis
Regression analysis
Researchers
Spin
Statistical learning
Systematic review
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In biomedical research, spin is the overinterpretation of findings, and it is a growing concern. To date, the presence of spin has not been evaluated in prognostic model research in oncology, including studies developing and validating models for individualized risk prediction. We conducted a systematic review, searching MEDLINE and EMBASE for oncology-related studies that developed and validated a prognostic model using machine learning published between 1st January, 2019, and 5th September, 2019. We used existing spin frameworks and described areas of highly suggestive spin practices. We included 62 publications (including 152 developed models; 37 validated models). Reporting was inconsistent between methods and the results in 27% of studies due to additional analysis and selective reporting. Thirty-two studies (out of 36 applicable studies) reported comparisons between developed models in their discussion and predominantly used discrimination measures to support their claims (78%). Thirty-five studies (56%) used an overly strong or leading word in their title, abstract, results, discussion, or conclusion. The potential for spin needs to be considered when reading, interpreting, and using studies that developed and validated prognostic models in oncology. Researchers should carefully report their prognostic model research using words that reflect their actual results and strength of evidence.
ISSN:0895-4356
1878-5921
DOI:10.1016/j.jclinepi.2023.03.012