Effective Phoneme Decoding With Hyperbolic Neural Networks for High-Performance Speech BCIs

Objective: Speech brain-computer interfaces (speech BCIs), which convert brain signals into spoken words or sentences, have demonstrated great potential for high-performance BCI communication. Phonemes are the basic pronunciation units. For monosyllabic languages such as Chinese Mandarin, where a wo...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on neural systems and rehabilitation engineering 2024, Vol.32, p.3432-3441
Main Authors: Tan, Xianhan, Lian, Qi, Zhu, Junming, Zhang, Jianmin, Wang, Yueming, Qi, Yu
Format: Article
Language:English
Subjects:
Accuracy
Adult
Algorithms
Brain-computer interface
Brain-Computer Interfaces
Cluster Analysis
Decoding
Electroencephalography - methods
Error analysis
Female
Humans
hyperbolic clustering
hyperbolic network
Kinematics
Language
Male
Neural activity
neural decoding
Neural Networks, Computer
Phonetics
Speech - physiology
speech BCIs
Vectors
Young Adult
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Objective: Speech brain-computer interfaces (speech BCIs), which convert brain signals into spoken words or sentences, have demonstrated great potential for high-performance BCI communication. Phonemes are the basic pronunciation units. For monosyllabic languages such as Chinese Mandarin, where a word usually contains less than three phonemes, accurate decoding of phonemes plays a vital role. We found that in the neural representation space, phonemes with similar pronunciations are often inseparable, leading to confusion in phoneme classification. Methods: We mapped the neural signals of phoneme pronunciation into a hyperbolic space for a more distinct phoneme representation. Critically, we proposed a hyperbolic hierarchical clustering approach to specifically learn a phoneme-level structure to guide the representation. Results: We found such representation facilitated greater distance between similar phonemes, effectively reducing confusion. In the phoneme decoding task, our approach demonstrated an average accuracy of 75.21% for 21 phonemes and outperformed existing methods across different experimental days. Conclusion: Our approach showed high accuracy in phoneme classification. By learning the phoneme-level neural structure, the representations of neural signals were more discriminative and interpretable. Significance: Our approach can potentially facilitate high-performance speech BCIs for Chinese and other monosyllabic languages.
ISSN:1534-4320
1558-0210
1558-0210
DOI:10.1109/TNSRE.2024.3457313