Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist

• Published in: JACC. Cardiovascular imaging 2020-09, Vol.13 (9), p.2017-2035
• Main Authors: Sengupta, Partho P., Shrestha, Sirish, Berthon, Béatrice, Messas, Emmanuel, Donal, Erwan, Tison, Geoffrey H., Min, James K., D’hooge, Jan, Voigt, Jens-Uwe, Dudley, Joel, Verjans, Johan W., Shameer, Khader, Johnson, Kipp, Lovstakken, Lasse, Tabassian, Mahdi, Piccirilli, Marco, Pernot, Mathieu, Yanamala, Naveena, Duchateau, Nicolas, Kagiyama, Nobuyuki, Bernard, Olivier, Slomka, Piotr, Deo, Rahul, Arnaout, Rima
• Format: Article
• Language: English
• Subjects: artificial intelligence; cardiovascular imaging; checklist; Computer Science; digital health; machine learning; Medical Imaging; reporting guidelines; reproducible research
• ISSN: 1936-878X; 1876-7591
• DOI: 10.1016/j.jcmg.2020.07.015

Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently published that introduce the fundamental principles and clinical application of ML for cardiologists. This paper builds on these introductory principles and outlines a more comprehensive list of crucial responsibilities that need to be completed when developing ML models. This paper aims to serve as a scientific foundation to aid investigators, data scientists, authors, editors, and reviewers involved in machine learning research with the intent of uniform reporting of ML investigations. An independent multidisciplinary panel of ML experts, clinicians, and statisticians worked together to review the theoretical rationale underlying 7 sets of requirements that may reduce algorithmic errors and biases. Finally, the paper summarizes a list of reporting items as an itemized checklist that highlights steps for ensuring correct application of ML models and the consistent reporting of model specifications and results. It is expected that the rapid pace of research and development and the increased availability of real-world evidence may require periodic updates to the checklist. [Display omitted] •Algorithm complexity and flexibility of ML techniques can result in inconsistencies in model reporting and interpretations.•The PRIME checklist provides 7 items to be reported for reducing algorithmic errors and biases.•The checklist aims to standardize reporting on model design, data, selection, assessment, evaluation, replicability, and limitations.•As artificial intelligence and ML technologies continue to grow, the checklist will need periodic updates.