Orthogonal Representations of Object Shape and Category in Deep Convolutional Neural Networks and Human Visual Cortex

Deep Convolutional Neural Networks (CNNs) are gaining traction as the benchmark model of visual object recognition, with performance now surpassing humans. While CNNs can accurately assign one image to potentially thousands of categories, network performance could be the result of layers that are tu...

Full description

Saved in:
Bibliographic Details
Published in:Scientific reports 2020-02, Vol.10 (1), p.2453, Article 2453
Main Authors: Zeman, Astrid A., Ritchie, J. Brendan, Bracci, Stefania, Op de Beeck, Hans
Format: Article
Language:English
Subjects:
59/36
631/114/2397
631/378/116/1925
631/378/2649/1723
Adult
Brain Mapping
Female
Functional magnetic resonance imaging
Humanities and Social Sciences
Humans
Male
multidisciplinary
Nerve Net - physiology
Neural networks
Neural Networks, Computer
Pattern recognition
Pattern Recognition, Visual
Photic Stimulation
Science
Science (multidisciplinary)
Temporal lobe
Visual cortex
Visual Cortex - physiology
Visual Pathways - physiology
Visual Perception
Visual system
Young Adult
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Deep Convolutional Neural Networks (CNNs) are gaining traction as the benchmark model of visual object recognition, with performance now surpassing humans. While CNNs can accurately assign one image to potentially thousands of categories, network performance could be the result of layers that are tuned to represent the visual shape of objects, rather than object category, since both are often confounded in natural images. Using two stimulus sets that explicitly dissociate shape from category, we correlate these two types of information with each layer of multiple CNNs. We also compare CNN output with fMRI activation along the human visual ventral stream by correlating artificial with neural representations. We find that CNNs encode category information independently from shape, peaking at the final fully connected layer in all tested CNN architectures. Comparing CNNs with fMRI brain data, early visual cortex (V1) and early layers of CNNs encode shape information. Anterior ventral temporal cortex encodes category information, which correlates best with the final layer of CNNs. The interaction between shape and category that is found along the human visual ventral pathway is echoed in multiple deep networks. Our results suggest CNNs represent category information independently from shape, much like the human visual system.
ISSN:2045-2322
2045-2322
DOI:10.1038/s41598-020-59175-0