Predicting dichotomised outcomes from high-dimensional data in biomedicine

• Published in: Journal of applied statistics 2024-07, Vol.51 (9), p.1756-1771
• Main Authors: Rauschenberger, Armin, Glaab, Enrico
• Format: Article
• Language: English
• Subjects: 62-04; binary classification; Biomedical materials; dichotomisation; Dimensional analysis; high-dimensional data; Linear/logistic regression; numerical prediction; Predictions; Regression; Regression analysis; ridge/lasso regularisation; Statistical analysis; Statistical methods
• ISSN: 0266-4763; 1360-0532
• DOI: 10.1080/02664763.2023.2233057

In many biomedical applications, we are more interested in the predicted probability that a numerical outcome is above a threshold than in the predicted value of the outcome. For example, it might be known that antibody levels above a certain threshold provide immunity against a disease, or a threshold for a disease severity score might reflect conversion from the presymptomatic to the symptomatic disease stage. Accordingly, biomedical researchers often convert numerical to binary outcomes (loss of information) to conduct logistic regression (probabilistic interpretation). We address this bad statistical practice by modelling the binary outcome with logistic regression, modelling the numerical outcome with linear regression, transforming the predicted values from linear regression to predicted probabilities, and combining the predicted probabilities from logistic and linear regression. Analysing high-dimensional simulated and experimental data, namely clinical data for predicting cognitive impairment, we obtain significantly improved predictions of dichotomised outcomes. Thus, the proposed approach effectively combines binary with numerical outcomes to improve binary classification in high-dimensional settings. An implementation is available in the R package cornet on GitHub ( https://github.com/rauschenberger/cornet ) and CRAN ( https://CRAN.R-project.org/package=cornet ).