Is word error rate a good indicator for spoken language understanding accuracy

It is a conventional wisdom in the speech community that better speech recognition accuracy is a good indicator for better spoken language understanding accuracy, given a fixed understanding component. The findings in this work reveal that this is not always the case. More important than word error...

Full description

Saved in:
Bibliographic Details
Main Authors: Ye-Yi Wang, Acero, A., Chelba, C.
Format: Conference Proceeding
Language:English
Subjects:
Credit cards
Error analysis
Hidden Markov models
Information systems
Natural languages
Robustness
Speech recognition
Testing
Training data
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:It is a conventional wisdom in the speech community that better speech recognition accuracy is a good indicator for better spoken language understanding accuracy, given a fixed understanding component. The findings in this work reveal that this is not always the case. More important than word error rate reduction, the language model for recognition should be trained to match the optimization objective for understanding. In this work, we applied a spoken language understanding model as the language model in speech recognition. The model was obtained with an example-based learning algorithm that optimized the understanding accuracy. Although the speech recognition word error rate is 46% higher than the trigram model, the overall slot understanding error can be reduced by as much as 17%.
DOI:10.1109/ASRU.2003.1318504