Brain-computer Interface Based on Motor Imagery with Visual Guidance and Its Application in Control of Simulated Unmanned Aerial Vehicle

• Published in: IEEE sensors journal 2024-04, Vol.24 (7), p.1-1
• Main Authors: Yan, Lirong, Yu, Hao, Liu, Yan, Xiang, Biao, Cheng, Yu, Xu, Jihong, Wu, Yibo, Yan, Fuwu
• Format: Article
• Language: English
• Subjects: Accuracy; Artificial neural networks; Brain-computer interface; Classification; Comparative studies; Convolutional neural network; Data analysis; Electroencephalography; Human-computer interface; Imagery; Modules; Motor imagery; Multi-branch structure; Paradigms; Side effects; Tracking control; Trajectory control; Unmanned aerial vehicle; Unmanned aerial vehicles; Visual effects; Visual Guidance
• ISSN: 1530-437X; 1558-1748
• DOI: 10.1109/JSEN.2024.3363754

Brain-computer interface (BCI) based on motor imagery (MI) supplies a communication method between human and machine independent of external stimulation. MI is based on the imagination quality of the subjects, which resulted in large variability and unreliability of the signals. Improved MI paradigms and data analysis methods would be helpful to reduce the side effect of the individual specificity and temporal variation, and increase the classification performance of MI-based BCI. Visual guidance could improve the MI quality, but there lacks a systematic analysis about the forms of the visual guidance and their effects on MI. In this paper, we conducted a comparative study to evaluate the MI paradigms with different visual guidance manners. Five paradigms were designed, i.e. pure MI, MI with synchronous pursuit (SP), asynchronous pursuit (AP), synchronous saccade (SS) and asynchronous saccade (AS). Furthermore, a convolutional neural network architecture with multiple receptive fields and attention module was proposed. Twenty subjects accomplished the experiments. The SP paradigm induced the most significant event-related desynchronization (ERD) phenomenon and the highest classification accuracy for electroencephalograph (EEG) signals. The proposed network achieved an average classification accuracy of 91.89% and standard deviation of 5.55%, which outperformed the compared methods. To test the applicability of the paradigm and the method, six subjects with different performance in the offline experiment then participated in an online experiment and a simulated brain-controlled trajectory tracking flight of the unmanned aerial vehicle (UAV). All the subjects could accomplish the task, and their performance was positively correlated with the classification accuracy and negatively correlated with the complexity of the tracking path. In general, the SP visual guidance effectively helped the subjects modulate their brain activity, leading to increased MI quality. The convolutional network with multiple receptive fields and an attention module showed the improved classification performance and robustness.