Evolution of the "Internet Plus Health Care" Mode Enabled by Artificial Intelligence: Development and Application of an Outpatient Triage System

• Published in: Journal of medical Internet research 2024-10, Vol.26 (3), p.e51711
• Main Authors: Yang, Lingrui, Pang, Jiali, Zuo, Song, Xu, Jian, Jin, Wei, Zuo, Feng, Xue, Kui, Xiao, Zhongzhou, Peng, Xinwei, Xu, Jie, Zhang, Xiaofan, Chen, Ruiyao, Luo, Shuqing, Zhang, Shaoting, Sun, Xin
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; China; Electronic Health Records; Hospitals; Humans; Internet; Medical care; Medical colleges; Medical records; Nurses; Original Paper; Outpatient services; Outpatients - statistics & numerical data; Patient satisfaction; Prospective Studies; Quality management; Technology application; Triage (Medicine); Triage - methods
• ISSN: 1438-8871; 1439-4456; 1438-8871
• DOI: 10.2196/51711

Although new technologies have increased the efficiency and convenience of medical care, patients still struggle to identify specialized outpatient departments in Chinese tertiary hospitals due to a lack of medical knowledge. The objective of our study was to develop a precise and subdividable outpatient triage system to improve the experiences and convenience of patient care. We collected 395,790 electronic medical records (EMRs) and 500 medical dialogue groups. The EMRs were divided into 3 data sets to design and train the triage model (n=387,876, 98%) and test (n=3957, 1%) and validate (n=3957, 1%) it. The triage system was altered based on the current BERT (Bidirectional Encoder Representations from Transformers) framework and evaluated by recommendation accuracies in Xinhua Hospital using the cancellation rates in 2021 and 2022, from October 29 to December 5. Finally, a prospective observational study containing 306 samples was conducted to compare the system's performance with that of triage nurses, which was evaluated by calculating precision, accuracy, recall of the top 3 recommended departments (recall@3), and time consumption. With 3957 (1%) records each, the testing and validation data sets achieved an accuracy of 0.8945 and 0.8941, respectively. Implemented in Xinhua Hospital, our triage system could accurately recommend 79 subspecialty departments and reduce the number of registration cancellations from 16,037 (3.83%) of the total 418,714 to 15,338 (3.53%) of the total 434200 (P