Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models

• Published in: Proceedings of the National Academy of Sciences - PNAS 2022-07, Vol.119 (27), p.1-e2115047119
• Main Authors: Jozwik, Kamila M., O’Keeffe, Jonathan, Storrs, Katherine R., Guo, Wenxuan, Golan, Tal, Kriegeskorte, Nikolaus
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Biological Sciences; Depth perception; Euclidean geometry; Face; Neural networks; Stimuli; Texture; Three dimensional models; Visual perception
• ISSN: 0027-8424; 1091-6490
• DOI: 10.1073/pnas.2115047119

Human vision is attuned to the subtle differences between individual faces. Yet we lack a quantitative way of predicting how similar two face images look and whether they appear to show the same person. Principal component–based three-dimensional (3D) morphable models are widely used to generate stimuli in face perception research. These models capture the distribution of real human faces in terms of dimensions of physical shape and texture. How well does a “face space” based on these dimensions capture the similarity relationships humans perceive among faces? To answer this, we designed a behavioral task to collect dissimilarity and same/different identity judgments for 232 pairs of realistic faces. Stimuli sampled geometric relationships in a face space derived from principal components of 3D shape and texture (Basel face model [BFM]). We then compared a wide range of models in their ability to predict the data, including the BFM from which faces were generated, an active appearance model derived from face photographs, and image-computable models of visual perception. Euclidean distance in the BFM explained both dissimilarity and identity judgments surprisingly well. In a comparison against 16 diverse models, BFM distance was competitive with representational distances in state-of-the-art deep neural networks (DNNs), including novel DNNs trained on BFM synthetic identities or BFM latents. Models capturing the distribution of face shape and texture across individuals are not only useful tools for stimulus generation. They also capture important information about how faces are perceived, suggesting that human face representations are tuned to the statistical distribution of faces.