Detection, Localization, and Sex Classification of Faces from Arbitrary Viewpoints and under Occlusion

• Published in: IEEE transactions on pattern analysis and machine intelligence 2009-09, Vol.31 (9), p.1567-1581
• Main Authors: Toews, M., Arbel, T.
• Format: Article
• Language: English
• Subjects: Age Factors; Algorithms; Applied sciences; Artificial Intelligence; Bayesian analysis; Bayesian methods; Classification; Computer science; control theory; systems; Data processing. List processing. Character string processing; Error analysis; Exact sciences and technology; Face - anatomy & histology; Face detection; faces; Female; Geometry; Humans; Image databases; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Intelligence; Male; Memory organisation. Data processing; Occlusion; Pattern analysis; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; probabilistic modeling; Reproducibility of Results; Robustness; Scale-invariant feature; Sensitivity and Specificity; Sex; sex classification; Sex Determination Analysis - methods; Sex Factors; Software; Solid modeling; Spatial databases; Speech and sound recognition and synthesis. Linguistics; Studies; Support vector machine classification; Support vector machines; viewpoint invariance; Visual; visual trait
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2008.233

This paper presents a novel framework for detecting, localizing, and classifying faces in terms of visual traits, e.g., sex or age, from arbitrary viewpoints and in the presence of occlusion. All three tasks are embedded in a general viewpoint-invariant model of object class appearance derived from local scale-invariant features, where features are probabilistically quantified in terms of their occurrence, appearance, geometry, and association with visual traits of interest. An appearance model is first learned for the object class, after which a Bayesian classifier is trained to identify the model features indicative of visual traits. The framework can be applied in realistic scenarios in the presence of viewpoint changes and partial occlusion, unlike other techniques assuming data that are single viewpoint, upright, prealigned, and cropped from background distraction. Experimentation establishes the first result for sex classification from arbitrary viewpoints, an equal error rate of 16.3 percent, based on the color FERET database. The method is also shown to work robustly on faces in cluttered imagery from the CMU profile database. A comparison with the geometry-free bag-of-words model shows that geometrical information provided by our framework improves classification. A comparison with support vector machines demonstrates that Bayesian classification results in superior performance.