Three myths about risk thresholds for prediction models

• Published in: BMC medicine 2019-10, Vol.17 (1), p.192-192, Article 192
• Main Authors: Wynants, Laure, van Smeden, Maarten, McLernon, David J, Timmerman, Dirk, Steyerberg, Ewout W, Van Calster, Ben
• Format: Article
• Language: English
• Subjects: Clinical risk prediction model; Data Interpretation, Statistical; Data science; Decision support techniques; Diagnosis; Forecasting - methods; Humans; Longitudinal Studies; Models, Statistical; Models, Theoretical; Mythology; Opinion; Risk; Risk Assessment - methods; Risk Assessment - standards; Threshold
• ISSN: 1741-7015; 1741-7015
• DOI: 10.1186/s12916-019-1425-3

Clinical prediction models are useful in estimating a patient's risk of having a certain disease or experiencing an event in the future based on their current characteristics. Defining an appropriate risk threshold to recommend intervention is a key challenge in bringing a risk prediction model to clinical application; such risk thresholds are often defined in an ad hoc way. This is problematic because tacitly assumed costs of false positive and false negative classifications may not be clinically sensible. For example, when choosing the risk threshold that maximizes the proportion of patients correctly classified, false positives and false negatives are assumed equally costly. Furthermore, small to moderate sample sizes may lead to unstable optimal thresholds, which requires a particularly cautious interpretation of results. We discuss how three common myths about risk thresholds often lead to inappropriate risk stratification of patients. First, we point out the contexts of counseling and shared decision-making in which a continuous risk estimate is more useful than risk stratification. Second, we argue that threshold selection should reflect the consequences of the decisions made following risk stratification. Third, we emphasize that there is usually no universally optimal threshold but rather that a plausible risk threshold depends on the clinical context. Consequently, we recommend to present results for multiple risk thresholds when developing or validating a prediction model. Bearing in mind these three considerations can avoid inappropriate allocation (and non-allocation) of interventions. Using discriminating and well-calibrated models will generate better clinical outcomes if context-dependent thresholds are used.