Modeling Task Engagement to Regulate Reinforcement Learning-based Decoding for Online Brain Control

• Published in: IEEE transactions on cognitive and developmental systems 2024-11, p.1-9
• Main Authors: Zhang, Xiang, Shen, Xiang, Wang, Yiwen
• Format: Article
• Language: English
• Subjects: brain control; Brain modeling; brain-machine interface; Decoding; engagement modulation; Kernel; kernel reinforcement learning; medial prefrontal cortex; Motors; Neural activity; online training; Rats; Time factors; Training; Training data; Trajectory
• ISSN: 2379-8920; 2379-8939
• DOI: 10.1109/TCDS.2024.3492199

Brain-Machine Interfaces (BMIs) offer significant promise for enabling paralyzed individuals to control external devices using their brain signals. One challenge is that during the online Brain Control (BC) process, subjects may not be completely immersed in the task, particularly when multiple steps are needed to achieve a goal. The decoder indiscriminately takes the less engaged trials as training data, which might decrease the decoding accuracy. In this paper, we propose an alternative kernel RL-based decoder that trains online with continuous parameter update. We model neural activity from the medial prefrontal cortex (mPFC), a reward-related brain region, to represent task engagement. This information is incorporated into a stochastic learning rate using an exponential model, which measures the relevancy of neural data. The proposed algorithm was evaluated in the experiment where rats performed a cursor-reaching BC task. We found the neural activities from mPFC contained the engagement information which was negatively correlated with trial response time. Moreover, compared to the RL method without task engagement modeling, our proposed method enhanced the training efficiency. It used half of the training data to achieve the same reconstruction accuracy of the cursor trajectory. The results demonstrate the potential of our RL framework for improving online brain control tasks.