Improving Speech Recognition for African American English with Audio Classification

Automatic speech recognition (ASR) systems have been shown to have large quality disparities between the language varieties they are intended or expected to recognize. One way to mitigate this is to train or fine-tune models with more representative datasets. But this approach can be hindered by lim...

Full description

Saved in:
Bibliographic Details
Main Authors: Garg, Shefali, Huo, Zhouyuan, Sim, Khe Chai, Schwartz, Suzan, Chua, Mason, Aksenova, Alena, Munkhdalai, Tsendsuren, King, Levi, Wright, Darryl, Mengesha, Zion, Hwang, Dongseong, Sainath, Tara, Beaufays, Francoise, Mengibar, Pedro Moreno
Format: Conference Proceeding
Language:English
Subjects:
African American English
automatic speech recognition
dialect classifier
equity
Error analysis
Robustness
Semisupervised learning
Signal processing
Speech recognition
Training
US English
Video on demand
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Automatic speech recognition (ASR) systems have been shown to have large quality disparities between the language varieties they are intended or expected to recognize. One way to mitigate this is to train or fine-tune models with more representative datasets. But this approach can be hindered by limited in-domain data for training and evaluation. We propose a new way to improve the robustness of a US English short-form speech recognizer using a small amount of out-of-domain (long-form) African American English (AAE) data. We use CORAAL, YouTube and Mozilla Common Voice to train an audio classifier to approximately output whether an utterance is AAE or some other variety including Mainstream American English (MAE). By combining the classifier output with coarse geographic information, we can select a subset of utterances from a large corpus of untranscribed short-form queries for semi-supervised learning at scale. Fine-tuning on this data results in a 38.5% relative word error rate disparity reduction between AAE and MAE without reducing MAE quality.
ISSN:2379-190X
DOI:10.1109/ICASSP48485.2024.10447116