Do face masks introduce bias in speech technologies? The case of automated scoring of speaking proficiency

The COVID-19 pandemic has led to a dramatic increase in the use of face masks worldwide. Face coverings can affect both acoustic properties of the signal as well as speech patterns and have unintended effects if the person wearing the mask attempts to use speech processing technologies. In this pape...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2020-11
Main Authors: Loukina, Anastassia, Keelan Evanini, Mulholland, Matthew, Blood, Ian, Zechner, Klaus
Format: Article
Language:English
Subjects:
Acoustic properties
Automation
Bias
COVID-19
English language
Language proficiency
Masks
Speaking
Speech processing
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Loukina, Anastassia
Keelan Evanini
Mulholland, Matthew
Blood, Ian
Zechner, Klaus
description The COVID-19 pandemic has led to a dramatic increase in the use of face masks worldwide. Face coverings can affect both acoustic properties of the signal as well as speech patterns and have unintended effects if the person wearing the mask attempts to use speech processing technologies. In this paper we explore the impact of wearing face masks on the automated assessment of English language proficiency. We use a dataset from a large-scale speaking test for which test-takers were required to wear face masks during the test administration, and we compare it to a matched control sample of test-takers who took the same test before the mask requirements were put in place. We find that the two samples differ across a range of acoustic measures and also show a small but significant difference in speech patterns. However, these differences do not lead to differences in human or automated scores of English language proficiency. Several measures of bias showed no differences in scores between the two groups.
doi_str_mv 10.48550/arxiv.2008.07520
format article
fullrecord <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2435010088</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2435010088</sourcerecordid><originalsourceid>FETCH-LOGICAL-a528-54b030c7b7cdf517db68a51d30cf21e06c505d519a26e0a8f3fbbedc2ab8b3ef3</originalsourceid><addsrcrecordid>eNotkM1LAzEQxYMgWGr_AG8Bz1snyWY3nkTqJxS89F4myaRNPzZ1syv635uil3nzfjDz4DF2I2BeG63hDvvv-DWXAGYOrZZwwSZSKVGZWsorNst5BwCyaaXWasJ2T4kHdMSPmPeZx27okx-LtxHPlucTkdvyoYwuHdImUn7gqy1xh5l4ChzHIR1xIM-zS33sNmdYrnB_3k99CtFF6tzPNbsMeMg0-9cpW708rxZv1fLj9X3xuKxQS1Pp2oIC19rW-aBF621jUAtfWJCCoHEatNfiHmVDgCaoYC15J9EaqyioKbv9e1uiP0fKw3qXxr4riWtZKw2iNGPULxVVWwM</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2435010088</pqid></control><display><type>article</type><title>Do face masks introduce bias in speech technologies? The case of automated scoring of speaking proficiency</title><source>Publicly Available Content (ProQuest)</source><source>Coronavirus Research Database</source><creator>Loukina, Anastassia ; Keelan Evanini ; Mulholland, Matthew ; Blood, Ian ; Zechner, Klaus</creator><creatorcontrib>Loukina, Anastassia ; Keelan Evanini ; Mulholland, Matthew ; Blood, Ian ; Zechner, Klaus</creatorcontrib><description>The COVID-19 pandemic has led to a dramatic increase in the use of face masks worldwide. Face coverings can affect both acoustic properties of the signal as well as speech patterns and have unintended effects if the person wearing the mask attempts to use speech processing technologies. In this paper we explore the impact of wearing face masks on the automated assessment of English language proficiency. We use a dataset from a large-scale speaking test for which test-takers were required to wear face masks during the test administration, and we compare it to a matched control sample of test-takers who took the same test before the mask requirements were put in place. We find that the two samples differ across a range of acoustic measures and also show a small but significant difference in speech patterns. However, these differences do not lead to differences in human or automated scores of English language proficiency. Several measures of bias showed no differences in scores between the two groups.</description><identifier>EISSN: 2331-8422</identifier><identifier>DOI: 10.48550/arxiv.2008.07520</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Acoustic properties ; Automation ; Bias ; COVID-19 ; English language ; Language proficiency ; Masks ; Speaking ; Speech processing</subject><ispartof>arXiv.org, 2020-11</ispartof><rights>2020. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/2435010088?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>780,784,25753,27925,37012,38516,43895,44590</link.rule.ids></links><search><creatorcontrib>Loukina, Anastassia</creatorcontrib><creatorcontrib>Keelan Evanini</creatorcontrib><creatorcontrib>Mulholland, Matthew</creatorcontrib><creatorcontrib>Blood, Ian</creatorcontrib><creatorcontrib>Zechner, Klaus</creatorcontrib><title>Do face masks introduce bias in speech technologies? The case of automated scoring of speaking proficiency</title><title>arXiv.org</title><description>The COVID-19 pandemic has led to a dramatic increase in the use of face masks worldwide. Face coverings can affect both acoustic properties of the signal as well as speech patterns and have unintended effects if the person wearing the mask attempts to use speech processing technologies. In this paper we explore the impact of wearing face masks on the automated assessment of English language proficiency. We use a dataset from a large-scale speaking test for which test-takers were required to wear face masks during the test administration, and we compare it to a matched control sample of test-takers who took the same test before the mask requirements were put in place. We find that the two samples differ across a range of acoustic measures and also show a small but significant difference in speech patterns. However, these differences do not lead to differences in human or automated scores of English language proficiency. Several measures of bias showed no differences in scores between the two groups.</description><subject>Acoustic properties</subject><subject>Automation</subject><subject>Bias</subject><subject>COVID-19</subject><subject>English language</subject><subject>Language proficiency</subject><subject>Masks</subject><subject>Speaking</subject><subject>Speech processing</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>COVID</sourceid><sourceid>PIMPY</sourceid><recordid>eNotkM1LAzEQxYMgWGr_AG8Bz1snyWY3nkTqJxS89F4myaRNPzZ1syv635uil3nzfjDz4DF2I2BeG63hDvvv-DWXAGYOrZZwwSZSKVGZWsorNst5BwCyaaXWasJ2T4kHdMSPmPeZx27okx-LtxHPlucTkdvyoYwuHdImUn7gqy1xh5l4ChzHIR1xIM-zS33sNmdYrnB_3k99CtFF6tzPNbsMeMg0-9cpW708rxZv1fLj9X3xuKxQS1Pp2oIC19rW-aBF621jUAtfWJCCoHEatNfiHmVDgCaoYC15J9EaqyioKbv9e1uiP0fKw3qXxr4riWtZKw2iNGPULxVVWwM</recordid><startdate>20201103</startdate><enddate>20201103</enddate><creator>Loukina, Anastassia</creator><creator>Keelan Evanini</creator><creator>Mulholland, Matthew</creator><creator>Blood, Ian</creator><creator>Zechner, Klaus</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>COVID</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20201103</creationdate><title>Do face masks introduce bias in speech technologies? The case of automated scoring of speaking proficiency</title><author>Loukina, Anastassia ; Keelan Evanini ; Mulholland, Matthew ; Blood, Ian ; Zechner, Klaus</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a528-54b030c7b7cdf517db68a51d30cf21e06c505d519a26e0a8f3fbbedc2ab8b3ef3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Acoustic properties</topic><topic>Automation</topic><topic>Bias</topic><topic>COVID-19</topic><topic>English language</topic><topic>Language proficiency</topic><topic>Masks</topic><topic>Speaking</topic><topic>Speech processing</topic><toplevel>online_resources</toplevel><creatorcontrib>Loukina, Anastassia</creatorcontrib><creatorcontrib>Keelan Evanini</creatorcontrib><creatorcontrib>Mulholland, Matthew</creatorcontrib><creatorcontrib>Blood, Ian</creatorcontrib><creatorcontrib>Zechner, Klaus</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>Coronavirus Research Database</collection><collection>ProQuest Central</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content (ProQuest)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering collection</collection><jtitle>arXiv.org</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Loukina, Anastassia</au><au>Keelan Evanini</au><au>Mulholland, Matthew</au><au>Blood, Ian</au><au>Zechner, Klaus</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Do face masks introduce bias in speech technologies? The case of automated scoring of speaking proficiency</atitle><jtitle>arXiv.org</jtitle><date>2020-11-03</date><risdate>2020</risdate><eissn>2331-8422</eissn><abstract>The COVID-19 pandemic has led to a dramatic increase in the use of face masks worldwide. Face coverings can affect both acoustic properties of the signal as well as speech patterns and have unintended effects if the person wearing the mask attempts to use speech processing technologies. In this paper we explore the impact of wearing face masks on the automated assessment of English language proficiency. We use a dataset from a large-scale speaking test for which test-takers were required to wear face masks during the test administration, and we compare it to a matched control sample of test-takers who took the same test before the mask requirements were put in place. We find that the two samples differ across a range of acoustic measures and also show a small but significant difference in speech patterns. However, these differences do not lead to differences in human or automated scores of English language proficiency. Several measures of bias showed no differences in scores between the two groups.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><doi>10.48550/arxiv.2008.07520</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2020-11
issn 2331-8422
language eng
recordid cdi_proquest_journals_2435010088
source Publicly Available Content (ProQuest); Coronavirus Research Database
subjects Acoustic properties
Automation
Bias
COVID-19
English language
Language proficiency
Masks
Speaking
Speech processing
title Do face masks introduce bias in speech technologies? The case of automated scoring of speaking proficiency
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-02T19%3A52%3A30IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Do%20face%20masks%20introduce%20bias%20in%20speech%20technologies?%20The%20case%20of%20automated%20scoring%20of%20speaking%20proficiency&rft.jtitle=arXiv.org&rft.au=Loukina,%20Anastassia&rft.date=2020-11-03&rft.eissn=2331-8422&rft_id=info:doi/10.48550/arxiv.2008.07520&rft_dat=%3Cproquest%3E2435010088%3C/proquest%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-a528-54b030c7b7cdf517db68a51d30cf21e06c505d519a26e0a8f3fbbedc2ab8b3ef3%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2435010088&rft_id=info:pmid/&rfr_iscdi=true