Development of Immersive Virtual Reality-Based Hand Rehabilitation System Using a Gesture-Controlled Rhythm Game With Vibrotactile Feedback: An fNIRS Pilot Study

• Published in: IEEE transactions on neural systems and rehabilitation engineering 2023-01, Vol.31, p.3732-3743
• Main Authors: Bae, Sungjin, Park, Hyung-Soon
• Format: Article
• Language: English
• Subjects: Brain; Color; Computer applications; Cortex (motor); Cortex (premotor); Cortex (somatosensory); Feedback; Functional near-infrared spectroscopy; Games; gesture recognition; Hand (anatomy); hand rehabilitation systems; Hemodynamic responses; Hemoglobin; Immersive virtual reality; Infrared spectra; Infrared spectroscopy; Medical imaging; Near infrared radiation; Neural plasticity; Prefrontal cortex; Rehabilitation; Rhythm; Sensory integration; Spectroscopy; Stroke; Stroke (medical condition); Supplementary motor area; Survival; Task analysis; Training; vibrotactile feedback; Virtual reality
• ISSN: 1534-4320; 1558-0210; 1558-0210
• DOI: 10.1109/TNSRE.2023.3312336

Recently, virtua reality (VR) has been widely utilized with rehabilitation to promote user engagement, which has been shown to induce brain plasticity. In this study, we developed a VR-based hand rehabilitation system consisting of a personalized gesture-controlled rhythm game with vibrotactile feedback and investigated the cortical activation pattern induced by our system using functional near-infrared spectroscopy (fNIRS). Our system provides vibrotactile feedback as the user matches their hand gestures to VR targets customized to their pre-recorded hand gestures. Cortical activation was measured via fNIRS during 420 seconds of alternating gameplay and rest in 11 healthy subjects and one stroke survivor. Regions of interest (ROI) were the prefrontal cortex (PFC), the premotor cortex & the supplementary motor area (PMC&SMA), the primary sensorimotor cortex (SM1), and the somatosensory association cortex (SAC). The mean success rate of gesture matching among healthy subjects was 90 % with a standard deviation of 10.7 %, and the success rate of the stroke survivor was 79.6 %. The averaged cortical activation map for the 11 healthy subjects and the individual cortical activation map for the single stroke survivor showed increased hemodynamic responses of oxygenated hemoglobin (HbO) during the VR-based hand rehabilitation compared to the resting condition. Paired t-test analysis demonstrated a significant increase in HbO activation values in 19 out of 51 channels, corresponding to all ROIs except the left PFC and PMC&SMA, which exhibited high subject variability. The experimental results indicate that the proposed system successfully activated brain areas related to motor planning/execution, multisensory integration, and attention.