Cyclic Transfer Learning for Mandarin-English Code-Switching Speech Recognition

• Published in: IEEE signal processing letters 2023-01, Vol.30, p.1-5
• Main Authors: Nga, Cao Hong, Vu, Duc-Quang, Luong, Huong Hoang, Huang, Chien-Lin, Wang, Jia-Ching
• Format: Article
• Language: English
• Subjects: code-switching speech recognition; cyclic transfer learning; Decoding; Learning; Performance enhancement; Speech coding; Speech recognition; Task analysis; Training; Transfer learning; Transformers
• ISSN: 1070-9908; 1558-2361
• DOI: 10.1109/LSP.2023.3307350

Transfer learning is a common method to improve the performance of the model on a target task via pre-training the model on pretext tasks. Different from the methods using monolingual corpora for pre-training, in this study, we propose a Cyclic Transfer Learning method (CTL) that utilizes both code-switching (CS) and monolingual speech resources as the pretext tasks. Moreover, the model in our approach is always alternately learned among these tasks. This helps our model can improve its performance via maintaining CS features during transferring knowledge. The experiment results on the standard SEAME Mandarin-English CS corpus have shown that our proposed CTL approach achieves the best performance with Mixed Error Rate (MER) of 16.3% on test_{man}, 24.1% on test_{sge}. In comparison to the baseline model that was pre-trained with monolingual data, our CTL method achieves 11.4% and 8.7% relative MER reduction on the test_{man} and test_{sge} sets, respectively. Besides, the CTL approach also outperforms compared to other state-of-the-art methods. The source code of the CTL method can be found at https://github.com/caohongnga/CTL-CSSR .