BCI-based hit-loop agent for human and AI robot co-learning with AIoT application

• Published in: Journal of ambient intelligence and humanized computing 2023-04, Vol.14 (4), p.3583-3607
• Main Authors: Lee, Chang-Shing, Wang, Mei-Hui, Kuan, Wen-Kai, Huang, Sheng-Hui, Tsai, Yi-Lin, Ciou, Zong-Han, Yang, Chen-Kang, Kubota, Naoyuki
• Format: Article
• Language: English
• Subjects: Agents (artificial intelligence); Artificial Intelligence; Buddhism; Cognition & reasoning; Computational Intelligence; Consciousness; Deep learning; Electroencephalography; Emotional intelligence; Emotions; Engineering; English language; Fuzzy logic; Fuzzy sets; Human-computer interface; Information processing; Knowledge bases (artificial intelligence); Language; Listening; Machine learning; Music; Neural networks; Ontology; Original Research; Particle swarm optimization; Reagents; Robotics and Automation; Robots; Students; Teaching; User Interfaces and Human Computer Interaction
• ISSN: 1868-5137; 1868-5145
• DOI: 10.1007/s12652-021-03487-0

In this paper, we propose a brain–computer interface (BCI)-based Human-in-the-Loop (Hit-Loop) agent for human and artificial intelligence (AI) colearning in music listening and appreciation with an Artificial Intelligence of Things (AIoT) application. The novel BCI-based Hit-Loop agent contains human intelligence with BCI-based AIoT-Fuzzy Markup Language (FML) and BCI-FML agents , as well as machine intelligence with AI-FML Hit-Loop and AIoT-FML agents . We used FML to facilitate communication between humans and the AI-FML robots through an AIoT-FML Learning Tool ( AIoT-FML-LT ), which was the core technology of the AI-FML Hit-Loop agent for the BCI-based music listening and appreciation application. Furthermore, the novel AIoT-FML-LT in conjunction with the BCI-FML and AIoT-FML agents was developed and presented for music listening and student learning. Moreover, the BCI-based AIoT-FML and AI-FML Hit-Loop agents were applied in English language learning, and the AIoT-FML-LT assisted in measuring student learning performance in Taiwan and Japan. The human-like high-level knowledge base and rule base were constructed by various domain experts, as well as the personalized electroencephalography (EEG) and student English learning data sets collected using the BCI device and AIoT-FML-LT , respectively, and were applied to machine learning models such as deep learning and particle swarm optimization. Additionally, the AIoT-FML-LT was connected to the AIoT-FML Hit-Loop agent for human and robot colearning. The relationship between human perceptions and the AIoT-FML Hit-Loop agent in terms of eyes , ears , nose , tongue , body , and brain corresponding to sights , sounds , smells , tastes , objects of touch , and mind are discussed. Finally, the students learned human language and AI language by using the AI-FML robots and AIoT-FML-LT together with the human English learning and AI-FML machine learning models, respectively. The experimental results reveal that the BCI-based Hit-Loop agent for human and AI-FML robot colearning in conjunction with AIoT applications can effectively facilitate music listening and appreciation application as well as English listening in Taiwan and Japan. The learning behavior and performance of the students also improved after incorporation of the human and robot colearning model.