COMO: Efficient Deep Neural Networks Expansion With COnvolutional MaxOut

In this paper, we extend the classic MaxOut strategy, originally designed for Multiple Layer Preceptors (MLPs), into CO nvolutional M ax O ut (COMO) - a new strategy making deep convolutional neural networks wider with parameter efficiency. Compared to the existing solutions, such as ResNeXt for Res...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on multimedia 2021, Vol.23, p.1722-1730
Main Authors: Zhao, Baoxin, Xiong, Haoyi, Bian, Jiang, Guo, Zhishan, Xu, Cheng-Zhong, Dou, Dejing
Format: Article
Language:English
Subjects:
Accuracy
Artificial neural networks
Classification
computational and artificial intelligence
Computer architecture
Convolution
Convolutional neural networks
Deep learning
Feature maps
feedforward neural networks
multilayer perceptrons
Neural networks
Parameters
Reduction
Spatial resolution
Transforms
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we extend the classic MaxOut strategy, originally designed for Multiple Layer Preceptors (MLPs), into CO nvolutional M ax O ut (COMO) - a new strategy making deep convolutional neural networks wider with parameter efficiency. Compared to the existing solutions, such as ResNeXt for ResNet or Inception for VGG-alikes, COMO works well on both linear architectures and the ones with skipped connections and residual blocks. More specifically, COMO adopts a novel split-transform-merge paradigm that extends the layers with spatial resolution reduction into multiple parallel splits. For the layer with COMO, each split passes the input feature maps through a 4D convolution operator with independent batch normalization operators for transformation, then merge into the aggregated output of the original sizes through max-pooling . Such a strategy is expected to tackle the potential classification accuracy degradation due to the spatial resolution reduction, by incorporating the multiple splits and max-pooling-based feature selection. Our experiment using a wide range of deep architectures shows that COMO can significantly improve the classification accuracy of ResNet/VGG-alike networks based on a large number of benchmark datasets. COMO further outperforms the existing solutions, e.g., Inceptions, ResNeXts, SE-ResNet, and Xception, that make networks wider, and it dominates in the comparison of accuracy versus parameter sizes.
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2020.3002614