A Unified Framework for Bidirectional Prototype Learning From Contaminated Faces Across Heterogeneous Domains

Existing heterogeneous face synthesis (HFS) methods focus on performing accurate image-to-image translation across domains, while they cannot effectively remove the nuisance facial variations such as poses, expressions or occlusions. To address such challenges, this paper studies a new practical het...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on information forensics and security 2022, Vol.17, p.1544-1557
Main Authors: Pang, Meng, Wang, Binghui, Huang, Siyu, Cheung, Yiu-Ming, Wen, Bihan
Format: Article
Language:English
Subjects:
adversarial learning
Coders
Contamination
Discriminators
Domains
Encoders-Decoders
Face recognition
Face synthesis
Faces
Generative adversarial networks
Generators
Hafnium
heterogeneous face recognition
heterogeneous prototype learning
Image reconstruction
Image synthesis
Learning
Prototypes
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Existing heterogeneous face synthesis (HFS) methods focus on performing accurate image-to-image translation across domains, while they cannot effectively remove the nuisance facial variations such as poses, expressions or occlusions. To address such challenges, this paper studies a new practical heterogeneous prototype learning (HPL) problem. To be specific, given a face image contaminated by facial variations from a source domain, HPL aims to reconstruct the variation-free prototype in a specified target domain. To tackle HPL, we propose a unified and end-to-end framework named bidirectional heterogeneous prototype learning (BHPL). As a bidirectional learning framework, BHPL is able to simultaneously reconstruct the heterogeneous prototypes across source-to-target as well as target-to-source domains. Furthermore, BHPL is capable of learning the identity prototype features for the contaminated face images from both source and target domains in order to perform robust heterogeneous face recognition. BHPL consists of an encoder-decoder structural generator and two dual-task discriminators, which play an adversarial game such that the generator learns the identity prototype feature and generates the cross-domain identity-preserved prototype for each input face image from both domains, and the discriminators accurately predict face identity and distinguish real versus fake prototypes. Empirically studies on multiple heterogeneous face datasets containing facial variations demonstrate the effectiveness of BHPL.
ISSN:1556-6013
1556-6021
DOI:10.1109/TIFS.2022.3164215