Investigations on sequence training of neural networks

In this paper we present an investigation of sequence-discriminative training of deep neural networks for automatic speech recognition. We evaluate different sequence-discriminative training criteria (MMI and MPE) and optimization algorithms (including SGD and Rprop) using the RASR toolkit. Further,...

Full description

Saved in:
Bibliographic Details
Main Authors: Wiesler, Simon, Golik, Pavel, Schluter, Ralf, Ney, Hermann
Format: Conference Proceeding
Language:English
Subjects:
deep neural networks
Hidden Markov models
Lattices
Neural networks
Optimization
sequence training
Speech
Speech recognition
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper we present an investigation of sequence-discriminative training of deep neural networks for automatic speech recognition. We evaluate different sequence-discriminative training criteria (MMI and MPE) and optimization algorithms (including SGD and Rprop) using the RASR toolkit. Further, we compare the training of the whole network with that of the output layer only. Technical details necessary for a robust training are studied, since there is no consensus yet on the ultimate training recipe. The investigation extends our previous work on training linear bottleneck networks from scratch showing the consistently positive effect of sequence training.
ISSN:1520-6149
2379-190X
DOI:10.1109/ICASSP.2015.7178835