Low Latency Acoustic Modeling Using Temporal Convolution and LSTMs

Bidirectional long short-term memory (BLSTM) acoustic models provide a significant word error rate reduction compared to their unidirectional counterpart, as they model both the past and future temporal contexts. However, it is nontrivial to deploy bidirectional acoustic models for online speech rec...

Full description

Saved in:
Bibliographic Details
Published in:IEEE signal processing letters 2018-03, Vol.25 (3), p.373-377
Main Authors: Peddinti, Vijayaditya, Wang, Yiming, Povey, Daniel, Khudanpur, Sanjeev
Format: Article
Language:English
Subjects:
Acoustic model
Acoustics
Computational modeling
Context
Context modeling
Convolution
long short-term memory (LSTM)
Neural networks
recurrent neural networks
Switches
time-delay neural networks
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Bidirectional long short-term memory (BLSTM) acoustic models provide a significant word error rate reduction compared to their unidirectional counterpart, as they model both the past and future temporal contexts. However, it is nontrivial to deploy bidirectional acoustic models for online speech recognition due to an increase in latency. In this letter, we propose the use of temporal convolution, in the form of time-delay neural network (TDNN) layers, along with unidirectional LSTM layers to limit the latency to 200 ms. This architecture has been shown to outperform the state-of-the-art low frame rate (LFR) BLSTM models. We further improve these LFR BLSTM acoustic models by operating them at higher frame rates at lower layers and show that the proposed model performs similar to these mixed frame rate BLSTMs. We present results on the Switchboard 300 h LVCSR task and the AMI LVCSR task, in the three microphone conditions.
ISSN:1070-9908
1558-2361
DOI:10.1109/LSP.2017.2723507