Developing and validating a knowledge-based AI assessment system for learning clinical core medical knowledge in otolaryngology

• Published in: Computers in biology and medicine 2024-08, Vol.178, p.108765, Article 108765
• Main Authors: Su, Jun-Ming, Hsu, Su-Yi, Fang, Te-Yung, Wang, Pa-Chun
• Format: Article
• Language: English
• Subjects: Acceptance tests; Adaptive assessment; Adaptive control; Adaptive systems; Artificial intelligence; Clinical core medical knowledge; COVID-19; Deep learning; Effectiveness; Knowledge aggregation; Knowledge based systems; Knowledge-based AI approaches; Machine learning; Online instruction; Otolaryngology; Teaching methods; Technology assessment; Training; User experience
• ISSN: 0010-4825; 1879-0534; 1879-0534
• DOI: 10.1016/j.compbiomed.2024.108765

Clinical core medical knowledge (CCMK) learning is essential for medical trainees. Adaptive assessment systems can facilitate self-learning, but extracting experts' CCMK is challenging, especially using modern data-driven artificial intelligence (AI) approaches (e.g., deep learning). This study aims to develop a multi-expert knowledge–aggregated adaptive assessment scheme (MEKAS) using knowledge-based AI approaches to facilitate the learning of CCMK in otolaryngology (CCMK-OTO) and validate its effectiveness through a one-month training program for CCMK-OTO education at a tertiary referral hospital. The MEKAS utilized the repertory grid technique and case-based reasoning to aggregate experts' knowledge to construct a representative CCMK base, thereby enabling adaptive assessment for CCMK-OTO training. The effects of longitudinal training were compared between the experimental group (EG) and the control group (CG). Both groups received a normal training program (routine meeting, outpatient/operation room teaching, and classroom teaching), while EG received MEKAS for self-learning. The EG comprised 22 UPGY trainees (6 postgraduate [PGY] and 16 undergraduate [UGY] trainees) and 8 otolaryngology residents (ENT-R); the CG comprised 24 UPGY trainees (8 PGY and 16 UGY trainees). The training effectiveness was compared through pre- and post-test CCMK-OTO scores, and user experiences were evaluated using a technology acceptance model-based questionnaire. Both UPGY (z = −3.976, P