Reconstructed spatial receptive field structures by reverse correlation technique explains the visual feature selectivity of units in deep convolutional neural networks

An important issue in dealing with Deep Convolutional Neural Networks (DCNN) is the 'black box problem', which represents the unknowns about internal information representation and processing, especially in the middle and higher layers. In this study, we adopted a systems neuroscience meth...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-03
Main Authors: Shiraishi, Yoshiyuki R, Sato, Hiromichi, Sanada, Takahisa M, Naito, Tomoyuki
Format: Article
Language:English
Subjects:
Artificial neural networks
Covariance
Neural networks
Selectivity
Stimuli
Visual fields
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:An important issue in dealing with Deep Convolutional Neural Networks (DCNN) is the 'black box problem', which represents the unknowns about internal information representation and processing, especially in the middle and higher layers. In this study, we adopted a systems neuroscience methodology to measure the visual feature selectivity and visualize the spatial receptive field of the units in VGG16. Orientation and spatial frequency tunings of each unit were measured using sinusoidal grating stimuli. The image category selectivity of each unit was also measured using natural image stimuli. The spatial structures of the receptive fields of all convolutional units were estimated by activation-weighted average (AWA) and activation-weighted covariance (AWC) analyses. In the middle layers (convolutional layers in block3 and block4), AWC analysis successfully reconstructed the receptive field that predicted the visual feature selectivity of the unit. Those results suggested the possibility that analyzing the reconstructed receptive field structure can be used to interpret the functional significance of the units and layers of a DCNN.
ISSN:2331-8422