Application of deep learning to the diagnosis of cervical lymph node metastasis from thyroid cancer with CT: external validation and clinical utility for resident training

• Published in: European radiology 2020-06, Vol.30 (6), p.3066-3072
• Main Authors: Lee, Jeong Hoon, Ha, Eun Ju, Kim, DaYoung, Jung, Yong Jun, Heo, Subin, Jang, Yong-ho, An, Sung Hyun, Lee, Kyungmin
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Artificial intelligence; Cancer; Computed tomography; Confidence intervals; Deep learning; Diagnosis; Diagnostic Radiology; Diagnostic systems; Head and Neck; Imaging; Internal Medicine; Interventional Radiology; Learning algorithms; Lymph nodes; Lymphatic system; Machine learning; Medical diagnosis; Medical imaging; Medicine; Medicine & Public Health; Metastases; Metastasis; Neuroradiology; Physicians; Radiology; Sensitivity; Thyroid; Thyroid cancer; Training; Ultrasound
• ISSN: 0938-7994; 1432-1084
• DOI: 10.1007/s00330-019-06652-4

Purpose This study aimed to validate a deep learning model’s diagnostic performance in using computed tomography (CT) to diagnose cervical lymph node metastasis (LNM) from thyroid cancer in a large clinical cohort and to evaluate the model’s clinical utility for resident training. Methods The performance of eight deep learning models was validated using 3838 axial CT images from 698 consecutive patients with thyroid cancer who underwent preoperative CT imaging between January and August 2018 (3606 and 232 images from benign and malignant lymph nodes, respectively). Six trainees viewed the same patient images ( n  = 242), and their diagnostic performance and confidence level (5-point scale) were assessed before and after computer-aided diagnosis (CAD) was included. Results The overall area under the receiver operating characteristics (AUROC) of the eight deep learning algorithms was 0.846 (range 0.784–0.884). The best performing model was Xception, with an AUROC of 0.884. The diagnostic accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of Xception were 82.8%, 80.2%, 83.0%, 83.0%, and 80.2%, respectively. After introducing the CAD system, underperforming trainees received more help from artificial intelligence than the higher performing trainees ( p  = 0.046), and overall confidence levels significantly increased from 3.90 to 4.30 ( p