Performance of a Deep Learning System and Performance of Optometrists for the Detection of Glaucomatous Optic Neuropathy Using Colour Retinal Photographs

• Published in: Bioengineering (Basel) 2024-11, Vol.11 (11), p.1139
• Main Authors: Jan, Catherine L, Vingrys, Algis, Henwood, Jacqueline, Shang, Xianwen, Davey, Christian, van Wijngaarden, Peter, Kong, George Y X, Fan Gaskin, Jennifer C, Soares Bezerra, Bernardo P, Stafford, Randall S, He, Mingguang
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Artificial intelligence; Artificial neural networks; Biobanks; Blindness; Classification; Color; Deep learning; Diagnosis; Glaucoma; glaucoma detection; Machine learning; Medical personnel; Neural networks; Optic neuropathy; Optometrists; Performance evaluation; Population studies; primary care; Research ethics; Retina; Sensitivity analysis; Statistical analysis
• ISSN: 2306-5354; 2306-5354
• DOI: 10.3390/bioengineering11111139

Glaucoma is the leading cause of irreversible blindness, with a significant proportion of cases remaining undiagnosed globally. The interpretation of optic disc and retinal nerve fibre layer images poses challenges for optometrists and ophthalmologists, often leading to misdiagnosis. AI has the potential to improve diagnosis. This study aims to validate an AI system (a convolutional neural network based on the Inception-v3 architecture) for detecting glaucomatous optic neuropathy (GON) using colour fundus photographs from a UK population and to compare its performance against Australian optometrists. A retrospective external validation study was conducted, comparing AI's performance with that of 11 AHPRA-registered optometrists in Australia on colour retinal photographs, evaluated against a reference (gold) standard established by a panel of glaucoma specialists. Statistical analyses were performed using sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC). For referable GON, the sensitivity of the AI (33.3% [95%CI: 32.4-34.3) was significantly lower than that of optometrists (65.1% [95%CI: 64.1-66.0]), < 0.0001, although with significantly higher specificity (AI: 97.4% [95%CI: 97.0-97.7]; optometrists: 85.5% [95%CI: 84.8-86.2], < 0.0001). The optometrists demonstrated significantly higher AUROC (0.753 [95%CI: 0.744-0.762]) compared to AI (0.654 [95%CI: 0.645-0.662], < 0.0001). The AI system exhibited lower performance than optometrists in detecting referable glaucoma. Our findings suggest that while AI can serve as a screening tool, both AI and optometrists have suboptimal performance for the nuanced diagnosis of glaucoma using fundus photographs alone. Enhanced training with diverse populations for AI is essential for improving GON detection and addressing the significant challenge of undiagnosed cases.