Foreign language acquisition via artificial intelligence and extended reality: design and evaluation

• Published in: Computer assisted language learning 2022-12, Vol.35 (9), p.2332-2360
• Main Authors: Divekar, Rahul R., Drozdal, Jaimie, Chabot, Samuel, Zhou, Yalun, Su, Hui, Chen, Yue, Zhu, Houming, Hendler, James A., Braasch, Jonas
• Format: Article
• Language: English
• Subjects: Achievement Gains; Artificial Intelligence; augmented; chatbots; Chinese; Chinese as a second language; Chinese languages; College Students; Colleges & universities; Comprehension; Computer Assisted Instruction; Computer assisted language learning; Computer Simulation; Conversation; conversational agents; Conversational Language Courses; Cooperative learning; Dyadic interaction; Education; Educational Environment; Experiential Learning; Feedback; foreign language acquisition; Foreign language learning; Foreign languages; Gamification; human-computer interaction; Immersion Programs; Language Acquisition; Language Tests; Language thought relationship; Learning; Learning environment; Listening Comprehension; Man Machine Systems; multi-modal interaction; multi-party interaction; Multimodality; Role Playing; Scores; Second language instruction; Second Language Learning; spoken dialogue systems; Student Experience; Students; virtual and extended reality; Vocabulary Development
• ISSN: 0958-8221; 1744-3210
• DOI: 10.1080/09588221.2021.1879162

Artificial Intelligence (AI) and Extended Reality (XR) have been employed in several foreign language education applications to increase the availability of experiential learning methods akin to international immersion programs. However, research in multi-modal spoken dialogue in L2 combined with immersive technologies and collaborative learning is thin, limiting students' experiences to solo interactions focused mostly on vocabulary and grammar in such settings. We intend to fill this gap as we present the Cognitive Immersive Language Learning Environment (CILLE). The AI in CILLE can hear, see, and understand its users and can engage with them in non-dyadic multimodal conversations. The XR offers students a feeling of being somewhere else without the use of intrusive devices and supports multi-party, multi-modal interactions. Together, AI and XR create naturalistic conversational interactions targeted towards comprehensive foreign language acquisition. We evaluate CILLE as a Chinese-as-a-foreign-language (CFL) education tool through a seven-week, mixed-methods study with university students (N = 10). Results display statistical significance and retained improvement in CFL vocabulary, comprehension, and conversation skills. Coupled with an analysis of student feedback and researcher observations, we show how CILLE is designed and experienced by students to learn CFL.