Deep FisherNet for Image Classification

Despite the great success of convolutional neural networks (CNNs) for the image classification task on data sets such as Cifar and ImageNet, CNN's representation power is still somewhat limited in dealing with images that have a large variation in size and clutter, where Fisher vector (FV) has...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2019-07, Vol.30 (7), p.2244-2250
Main Authors: Tang, Peng, Wang, Xinggang, Shi, Baoguang, Bai, Xiang, Liu, Wenyu, Tu, Zhuowen
Format: Article
Language:English
Subjects:
Aggregates
Artificial neural networks
Back propagation networks
Classification
Clutter
Coding
Computer applications
Convolutional neural networks (CNNs)
end to end
Feature extraction
Fisher layer
Fisher vector (FV)
Image classification
Image representation
Learning systems
Neural networks
Probabilistic models
Support vector machines
Task analysis
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Despite the great success of convolutional neural networks (CNNs) for the image classification task on data sets such as Cifar and ImageNet, CNN's representation power is still somewhat limited in dealing with images that have a large variation in size and clutter, where Fisher vector (FV) has shown to be an effective encoding strategy. FV encodes an image by aggregating local descriptors with a universal generative Gaussian mixture model (GMM). FV, however, has limited learning capability and its parameters are mostly fixed after constructing the codebook. To combine together the best of the two worlds, we propose in this brief a neural network structure with FV layer being part of an end-to-end trainable system that is differentiable; we name our network FisherNet that is learnable using back propagation. Our proposed FisherNet combines CNN training and FV encoding in a single end-to-end structure. We observe a clear advantage of FisherNet over plain CNN and standard FV in terms of both classification accuracy and computational efficiency on the challenging PASCAL visual object classes object classification and emotion image classification tasks.
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2018.2874657