Evaluation of word confidence for speech recognition systems

Confidence measures enable us to assess the output of a speech recognition system. The confidence measure provides us with an estimate of the probability that a word in the recognizer output is either correct or incorrect. In this paper we discuss ways in which to quantify the performance of confide...

Full description

Saved in:
Bibliographic Details
Published in:Computer speech & language 1999-10, Vol.13 (4), p.299-319
Main Authors: Siu, Manhung, Gish, Herbert
Format: Article
Language:English
Subjects:
Linguistics
Mathematics and linguistics
Quantitative studies
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Confidence measures enable us to assess the output of a speech recognition system. The confidence measure provides us with an estimate of the probability that a word in the recognizer output is either correct or incorrect. In this paper we discuss ways in which to quantify the performance of confidence measures in terms of their discrimination power and bias. In particular, we analyze two different performance metrics: the classification equal error rate and the normalized mutual information metric. We then report experimental results of using these metrics to compare four different confidence measure estimation schemes. We also discuss the relationship between these metrics and the operating point of the speech recognition system and develop an approach to the robust estimation of normalized mutual information.
ISSN:0885-2308
1095-8363
DOI:10.1006/csla.1999.0126