AI-doscopist: a real-time deep-learning-based algorithm for localising polyps in colonoscopy videos with edge computing devices

• Published in: NPJ digital medicine 2020-05, Vol.3 (1), p.73-8, Article 73
• Main Authors: Poon, Carmen C. Y., Jiang, Yuqi, Zhang, Ruikai, Lo, Winnie W. Y., Cheung, Maggie S. H., Yu, Ruoxi, Zheng, Yali, Wong, John C. T., Liu, Qing, Wong, Sunny H., Mak, Tony W. C., Lau, James Y. W.
• Format: Article
• Language: English
• Subjects: 692/308/575; 692/699/67; Artificial intelligence; Biomedicine; Biotechnology; Colonoscopy; Deep learning; Digital technology; Health informatics; Medicine; Medicine & Public Health; Polyps
• ISSN: 2398-6352; 2398-6352
• DOI: 10.1038/s41746-020-0281-z

We have designed a deep-learning model, an “Artificial Intelligent Endoscopist (a.k.a. AI-doscopist)”, to localise colonic neoplasia during colonoscopy. This study aims to evaluate the agreement between endoscopists and AI-doscopist for colorectal neoplasm localisation. AI-doscopist was pre-trained by 1.2 million non-medical images and fine-tuned by 291,090 colonoscopy and non-medical images. The colonoscopy images were obtained from six databases, where the colonoscopy images were classified into 13 categories and the polyps’ locations were marked image-by-image by the smallest bounding boxes. Seven categories of non-medical images, which were believed to share some common features with colorectal polyps, were downloaded from an online search engine. Written informed consent were obtained from 144 patients who underwent colonoscopy and their full colonoscopy videos were prospectively recorded for evaluation. A total of 128 suspicious lesions were resected or biopsied for histological confirmation. When evaluated image-by-image on the 144 full colonoscopies, the specificity of AI-doscopist was 93.3%. AI-doscopist were able to localise 124 out of 128 polyps (polyp-based sensitivity = 96.9%). Furthermore, after reviewing the suspected regions highlighted by AI-doscopist in a 102-patient cohort, an endoscopist has high confidence in recognizing four missed polyps in three patients who were not diagnosed with any lesion during their original colonoscopies. In summary, AI-doscopist can localise 96.9% of the polyps resected by the endoscopists. If AI-doscopist were to be used in real-time, it can potentially assist endoscopists in detecting one more patient with polyp in every 20–33 colonoscopies.