Accuracy Comparison Across Face Recognition Algorithms: Where Are We on Measuring Race Bias?

Previous generations of face recognition algorithms differ in accuracy for images of different races (race bias). Here, we present the possible underlying factors (data-driven and scenario modeling) and methodological considerations for assessing race bias in algorithms. We discuss data-driven facto...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on biometrics, behavior, and identity science behavior, and identity science, 2021-01, Vol.3 (1), p.101-111
Main Authors: Cavazos, Jacqueline G., Phillips, P. Jonathon, Castillo, Carlos D., O'Toole, Alice J.
Format: Article
Language:English
Subjects:
Accuracy
Algorithms
Artificial neural networks
Bias
Computational modeling
Convolutional neural networks
deep convolutional neural networks
Demographics
Face recognition
Face recognition algorithm
Faces
Image quality
Modelling
Object recognition
Population statistics
Prediction algorithms
Principal component analysis
Race
race bias
the other-race effect
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Previous generations of face recognition algorithms differ in accuracy for images of different races (race bias). Here, we present the possible underlying factors (data-driven and scenario modeling) and methodological considerations for assessing race bias in algorithms. We discuss data-driven factors (e.g., image quality, image population statistics, and algorithm architecture), and scenario modeling factors that consider the role of the "user" of the algorithm (e.g., threshold decisions and demographic constraints). To illustrate how these issues apply, we present data from four face recognition algorithms (a previous-generation algorithm and three deep convolutional neural networks, DCNNs) for East Asian and Caucasian faces. First, dataset difficulty affected both overall recognition accuracy and race bias, such that race bias increased with item difficulty. Second, for all four algorithms, the degree of bias varied depending on the identification decision threshold. To achieve equal false accept rates (FARs), East Asian faces required higher identification thresholds than Caucasian faces, for all algorithms. Third, demographic constraints on the formulation of the distributions used in the test, impacted estimates of algorithm accuracy. We conclude that race bias needs to be measured for individual applications and we provide a checklist for measuring this bias in face recognition algorithms.
ISSN:2637-6407
2637-6407
DOI:10.1109/TBIOM.2020.3027269