Mandarin Electrolaryngeal Speech Recognition Based on WaveNet-CTC

• Published in: Journal of speech, language, and hearing research language, and hearing research, 2019-07, Vol.62 (7), p.2203-2212
• Main Authors: Qian, Zhaopeng, Wang, Li, Zhang, Shaochuan, Liu, Chan, Niu, Haijun
• Format: Article
• Language: English
• Subjects: Accuracy; Acoustics; Affricates; Algorithms; Analysis; Artificial Intelligence; Augmentative and Alternative Communication; Automation; Chinese; Deep learning; Fundamental frequency; Intelligibility; Laryngectomy; Laryngology; Mandarin; Mandarin Chinese; Maximum Likelihood Statistics; Methods; Networks; Noise; Performance evaluation; Phonemes; Phonemics; Radiation (Physics); Recognition (Psychology); Researchers; Romanization; Sentences; Signal processing; Speech; Speech recognition; Voice recognition
• ISSN: 1092-4388; 1558-9102
• DOI: 10.1044/2019_JSLHR-S-18-0313

Purpose: The application of Chinese Mandarin electrolaryngeal (EL) speech for laryngectomees has been limited by its drawbacks such as single fundamental frequency, mechanical sound, and large radiation noise. To improve the intelligibility of Chinese Mandarin EL speech, a new perspective using the automatic speech recognition (ASR) system was proposed, which can convert EL speech into healthy speech, if combined with text-to-speech. Method: An ASR system was designed to recognize EL speech based on a deep learning model WaveNet and the connectionist temporal classification (WaveNet-CTC). This system mainly consists of 3 parts: the acoustic model, the language model, and the decoding model. The acoustic features are extracted during speech preprocessing, and 3,230 utterances of EL speech mixed with 10,000 utterances of healthy speech are used to train the ASR system. Comparative experiment was designed to evaluate the performance of the proposed method. Results: The results show that the proposed ASR system has higher stability and generalizability compared with the traditional methods, manifesting superiority in terms of Chinese characters, Chinese words, short sentences, and long sentences. Phoneme confusion occurs more easily in the stop and affricate of EL speech than the healthy speech. However, the highest accuracy of the ASR could reach 83.24% when 3,230 utterances of EL speech were used to train the ASR system. Conclusions: This study indicates that EL speech could be recognized effectively by the ASR based on WaveNet-CTC. This proposed method has a higher generalization performance and better stability than the traditional methods. A higher accuracy of the ASR system based on WaveNet-CTC can be obtained, which means that EL speech can be converted into healthy speech.