Artificial intelligence in virtual standardized patients: Combining natural language understanding and rule based dialogue management to improve conversational fidelity

• Published in: Medical teacher 2023-03, Vol.45 (3), p.279-285
• Main Authors: Maicher, Kellen R., Stiff, Adam, Scholl, Marisa, White, Michael, Fosler-Lussier, Eric, Schuler, William, Serai, Prashant, Sunder, Vishal, Forrestal, Hannah, Mendella, Lexi, Adib, Mahsa, Bratton, Camille, Lee, Kevin, Danforth, Douglas R.
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Artificial intelligence; Automation; Communication skills; Fidelity; First year; Humans; Medical students; Natural language; Patients; simulation; Speech recognition; Standardized patients; Trainees
• ISSN: 0142-159X; 1466-187X
• DOI: 10.1080/0142159X.2022.2130216

Advances in natural language understanding have facilitated the development of Virtual Standardized Patients (VSPs) that may soon rival human patients in conversational ability. We describe herein the development of an artificial intelligence (AI) system for VSPs enabling students to practice their history taking skills. Our system consists of (1) Automated Speech Recognition (ASR), (2) hybrid AI for question identification, (3) classifier to choose between the two systems, and (4) automated speech generation. We analyzed the accuracy of the ASR, the two AI systems, the classifier, and student feedback with 620 first year medical students from 2018 to 2021. System accuracy improved from ∼75% in 2018 to ∼90% in 2021 as refinements in algorithms and additional training data were utilized. Student feedback was positive, and most students felt that practicing with the VSPs was a worthwhile experience. We have developed a novel hybrid dialogue system that enables artificially intelligent VSPs to correctly answer student questions at levels comparable with human SPs. This system allows trainees to practice and refine their history-taking skills before interacting with human patients.