Augmented Sensory Feedback during Training of Upper Extremity Function in Virtual Reality

• Main Authors: Shi, Yu, Liu, Mingxiao, Dewil, Sophie, Harel, Noam Y., Sanford, Sean, Nataraj, Raviraj
• Format: Conference Proceeding
• Language: English
• Subjects: augmented sensory feedback; Cognitive load; motor rehabilitation; Motors; Physiology; Robot sensing systems; Training; Virtual reality; Visualization
• ISSN: 2372-9198
• DOI: 10.1109/CBMS61543.2024.00046

This study examined how varying the type (visual, haptic, or visual plus haptic) of augmented sensory feedback (ASF) cues used for guidance during motor training with virtual reality (VR) can uniquely impact performance and physiological responses indicative of cognitive load and physical arousal. This work aims to demonstrate the need and opportunity to optimize VR approaches for motor rehabilitation based on how the ASF is provided. This study utilized a custom VR platform for rehabilitating upper-body function for a myoelectric control task. Neurotypical participants (n=15) exerted near-isometric muscular exertions, and the resultant electromyographic (EMG) recordings were input to a support vector machine for commanding movement of a VR robot arm tasked to contact various targets. Motor performance was based on minimizing the motion pathlength of the robot end-effector and minimizing the time to complete each trial. Skin-surface electroencephalography (EEG) and electrodermal activity (EDA) were measured to assess cognitive loading and physical arousal, respectively. For this task, multimodal ASF (i.e., visual plus haptic cues) generated the highest cognitive load as expected; however, visual ASF produced more arousal and most improved post-training performance. Our findings suggest how participant performance and physiological states are related and governable by ASF. Thus, computerized approaches, such as VR, for motor training may be optimized based on how sensory-driven guidance cues are delivered.