Source Aware Deep Learning Framework for Hand Kinematic Reconstruction Using EEG Signal

• Published in: IEEE transactions on cybernetics 2023-07, Vol.53 (7), p.4094-4106
• Main Authors: Pancholi, Sidharth, Giri, Amita, Jain, Anant, Kumar, Lalan, Roy, Sitikantha
• Format: Article
• Language: English
• Subjects: Actuators; Artificial neural networks; Brain modeling; Brain-computer interface (BCI); Convolutional neural networks; Correlation coefficients; Deep learning; Electroencephalography; electroencephalography (EEG); Exoskeletons; Human-computer interface; intention mapping; Kinematics; Location awareness; Machine learning; motion trajectory prediction (MTP); Multilayer perceptrons; noninvasive; Reconstruction; source localization; State of the art; Trajectory; Trajectory analysis; Wavelet transforms
• ISSN: 2168-2267; 2168-2275
• DOI: 10.1109/TCYB.2022.3166604

The ability to reconstruct the kinematic parameters of hand movement using noninvasive electroencephalography (EEG) is essential for strength and endurance augmentation using exoskeleton/exosuit. For system development, the conventional classification-based brain-computer interface (BCI) controls external devices by providing discrete control signals to the actuator. A continuous kinematic reconstruction from EEG signal is better suited for practical BCI applications. The state-of-the-art multivariable linear regression (mLR) method provides a continuous estimate of hand kinematics, achieving a maximum correlation of up to 0.67 between the measured and the estimated hand trajectory. In this work, three novel source aware deep learning models are proposed for motion trajectory prediction (MTP). In particular, multilayer perceptron (MLP), convolutional neural network-long short-term memory (CNN-LSTM), and wavelet packet decomposition (WPD) for CNN-LSTM are presented. In addition, novelty in the work includes the utilization of brain source localization (BSL) [using standardized low-resolution brain electromagnetic tomography (sLORETA)] for the reliable decoding of motor intention. The information is utilized for channel selection and accurate EEG time segment selection. The performance of the proposed models is compared with the traditionally utilized mLR technique on the reach, grasp, and lift (GAL) dataset. The effectiveness of the proposed framework is established using the Pearson correlation coefficient (PCC) and trajectory analysis. A significant improvement in the correlation coefficient is observed when compared with the state-of-the-art mLR model. Our work bridges the gap between the control and the actuator block, enabling real-time BCI implementation.