Systematic comparison of machine learning algorithms to develop and validate predictive models for periodontitis

• Published in: Journal of clinical periodontology 2022-10, Vol.49 (10), p.958-969
• Main Authors: Bashir, Nasir Z., Rahman, Zahid, Chen, Sam Li‐Sheng
• Format: Article
• Language: English
• Subjects: Algorithms; computing; Diagnosis, Epidemiology and Associated Co‐morbidities; Gum disease; Humans; Learning algorithms; Machine Learning; Original; Periodontitis; Periodontitis - diagnosis; Prediction models; predictive modelling; Predictive Value of Tests; ROC Curve; statistics
• ISSN: 0303-6979; 1600-051X
• DOI: 10.1111/jcpe.13692

Aim The aim of this study was to compare the validity of different machine learning algorithms to develop and validate predictive models for periodontitis. Materials and Methods Using national survey data from Taiwan (n = 3453) and the United States (n = 3685), predictors of periodontitis were extracted from the datasets and pre‐processed, and then 10 machine learning algorithms were trained to develop predictive models. The models were validated both internally (bootstrap sampling) and externally (alternative country's dataset). The algorithms were compared across six performance metrics ([i] area under the curve for the receiver operating characteristic [AUC], [ii] accuracy, [iii] sensitivity, [iv] specificity, [v] positive predictive value, and [vi] negative predictive value) and two methods of data pre‐processing ([i] machine‐learning‐based feature selection and [ii] dimensionality reduction into principal components). Results Many algorithms showed extremely strong performance during internal validation (AUC > 0.95, accuracy > 95%). However, this was not replicated in external validation, where predictive performance of all algorithms dropped off drastically. Furthermore, predictive performance differed according to data pre‐processing methodology and the cohort on which they were trained. Conclusions Larger sample sizes and more complex predictors of periodontitis are required before machine learning can be leveraged to its full potential.