A comparison of ASR and human errors for transcription of non-native spontaneous speech

In this paper, we compare ASR and human transcriptions of non-native speech to investigate to what extent the accuracy and the patterns of errors of a modern ASR system match those of human listeners in the context of automated assessment of L2 English language proficiency. We obtained multiple naï...

Full description

Saved in:
Bibliographic Details
Main Authors: Mulholland, Matthew, Lopez, Melissa, Evanini, Keelan, Loukina, Anastassia, Yao Qian
Format: Conference Proceeding
Language:English
Subjects:
Assessments
automatic speech recognition
Conferences
Context
Correlation
Crowdsourcing
Electronics
Error analysis
Fragments
L2 speech
Recall
Speech
Speech recognition
speech transcription
Spontaneous
Syntactics
Vocabulary
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we compare ASR and human transcriptions of non-native speech to investigate to what extent the accuracy and the patterns of errors of a modern ASR system match those of human listeners in the context of automated assessment of L2 English language proficiency. We obtained multiple naïve transcriptions of short fragments of non-native spontaneous speech with different proficiency levels using crowdsourcing and matched these against the output of an ASR system. We compare WER and recall at the fragment level and consider human-ASR agreement at the word level. We find that we are able to attain a commensurate level of transcription quality using ASR, but the patterns of errors between the two groups differ at the word level.
ISSN:2379-190X
DOI:10.1109/ICASSP.2016.7472800