Reverberation Model-Based Decoding in the Logmelspec Domain for Robust Distant-Talking Speech Recognition

• Published in: IEEE transactions on audio, speech, and language processing speech, and language processing, 2010-09, Vol.18 (7), p.1676-1691
• Main Authors: Sehr, Armin, Maas, Roland, Kellermann, Walter
• Format: Article
• Language: English
• Subjects: Acoustic modeling; Algorithms; Automatic speech recognition; Constraint optimization; Contracts; Decoding; distant-talking automatic speech recognition (ASR); Estimates; Hidden Markov models; Loudspeakers; Mathematical models; Microphones; model-based dereverberation; Nonlinearity; Optimization; Recognition; Reverberation; reverberation model; robust ASR; Robustness; Speech recognition; Studies; Training; Voice recognition
• ISSN: 1558-7916; 2329-9290; 1558-7924; 2329-9304
• DOI: 10.1109/TASL.2010.2050511

The REMOS (REverberation MOdeling for Speech recognition) concept for reverberation-robust distant-talking speech recognition, introduced in "Distant-talking continuous speech recognition based on a novel reverberation model in the feature domain" (A. Sehr , in Proc. Interspeech, 2006, pp. 769-772) for melspectral features, is extended to logarithmic melspectral (logmelspec) features in this contribution. Thus, the favorable properties of REMOS, including its high flexibility with respect to changing reverberation conditions, become available in the more competitive logmelspec domain. Based on a combined acoustic model consisting of a hidden Markov model (HMM) network and a reverberation model (RM), REMOS determines clean-speech and reverberation estimates during recognition. Therefore, in each iteration of a modified Viterbi algorithm, an inner optimization operation maximizes the joint density of the current HMM output and the RM output subject to the constraint that their combination is equal to the current reverberant observation. Since the combination operation in the logmelspec domain is nonlinear, numerical methods appear necessary for solving the constrained inner optimization problem. A novel reformulation of the constraint, which allows for an efficient solution by nonlinear optimization algorithms, is derived in this paper so that a practicable implementation of REMOS for logmelspec features becomes possible. An in-depth analysis of this REMOS implementation investigates the statistical properties of its reverberation estimates and thus derives possibilities for further improving the performance of REMOS. Connected digit recognition experiments show that the proposed REMOS version in the logmelspec domain significantly outperforms the melspec version. While the proposed RMs with parameters estimated by straightforward training for a given room are robust to a mismatch of the speaker-microphone distance, their performance significantly decreases if they are used in a room with substantially different conditions. However, by training multi-style RMs with data from several rooms, good performance can be achieved across different rooms.