Multilingual deep neural network based acoustic modeling for rapid language adaptation

This paper presents a study on multilingual deep neural network (DNN) based acoustic modeling and its application to new languages. We investigate the effect of phone merging on multilingual DNN in context of rapid language adaptation. Moreover, the combination of multilingual DNNs with Kullback-Lei...

Full description

Saved in:
Bibliographic Details
Main Authors: Ngoc Thang Vu, Imseng, David, Povey, Daniel, Motlicek, Petr, Schultz, Tanja, Bourlard, Herve
Format: Conference Proceeding
Language:English
Subjects:
Acoustics
Decoding
Hidden Markov models
KL-HMM
Multilingual DNN
Neural networks
phone merging
rapid language adaptation
Speech
Training
Training data
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper presents a study on multilingual deep neural network (DNN) based acoustic modeling and its application to new languages. We investigate the effect of phone merging on multilingual DNN in context of rapid language adaptation. Moreover, the combination of multilingual DNNs with Kullback-Leibler divergence based acoustic modeling (KL-HMM) is explored. Using ten different languages from the Globalphone database, our studies reveal that crosslingual acoustic model transfer through multilingual DNNs is superior to unsupervised RBM pre-training and greedy layer-wise supervised training. We also found that KL-HMM based decoding consistently outperforms conventional hybrid decoding, especially in low-resource scenarios. Furthermore, the experiments indicate that multilingual DNN training equally benefits from simple phoneset concatenation and manually derived universal phonesets.
ISSN:1520-6149
2379-190X
DOI:10.1109/ICASSP.2014.6855086