PASS: Protected Attribute Suppression System for Mitigating Bias in Face Recognition

• Main Authors: Dhar, Prithviraj, Gleason, Joshua, Roy, Aniket, Castillo, Carlos D., Chellappa, Rama
• Format: Conference Proceeding
• Language: English
• Subjects: accountability; and ethics in vision; Biometrics; Computer vision; Encoding; Face recognition; Faces; Fairness; Open area test sites; Privacy; Training; transparency
• ISSN: 2380-7504
• DOI: 10.1109/ICCV48922.2021.01481

Face recognition networks encode information about sensitive attributes while being trained for identity classification. Such encoding has two major issues: (a) it makes the face representations susceptible to privacy leakage (b) it appears to contribute to bias in face recognition. However, existing bias mitigation approaches generally require end-to-end training and are unable to achieve high verification accuracy. Therefore, we present a descriptor-based adversarial de-biasing approach called 'Protected Attribute Suppression System (PASS)'. PASS can be trained on top of descriptors obtained from any previously trained high-performing network to classify identities and simultaneously reduce encoding of sensitive attributes. This eliminates the need for end-to-end training. As a component of PASS, we present a novel discriminator training strategy that discourages a network from encoding protected attribute information. We show the efficacy of PASS to reduce gender and skintone information in descriptors from SOTA face recognition networks like Arcface. As a result, PASS descriptors outperform existing baselines in reducing gender and skintone bias on the IJB-C dataset, while maintaining a high verification accuracy.