A dynamic CNN pruning method based on matrix similarity

Network pruning is one of the predominant approaches for deep model compression. Pruning large neural networks while maintaining their performance is often desirable because space and time complexity are reduced. Current pruning methods mainly focus on the importance of filters in the whole task. Di...

Full description

Saved in:
Bibliographic Details
Published in:Signal, image and video processing image and video processing, 2021-03, Vol.15 (2), p.381-389
Main Authors: Shao, Mingwen, Dai, Junhui, Kuang, Jiandong, Meng, Deyu
Format: Article
Language:English
Subjects:
Compression ratio
Computer Imaging
Computer Science
Feature maps
Image Processing and Computer Vision
Multimedia Information Systems
Original Paper
Pattern Recognition and Graphics
Pruning
Signal,Image and Speech Processing
Similarity
Virtual networks
Vision
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Network pruning is one of the predominant approaches for deep model compression. Pruning large neural networks while maintaining their performance is often desirable because space and time complexity are reduced. Current pruning methods mainly focus on the importance of filters in the whole task. Different from previous methods, this paper focuses on the similarity between the filters or feature maps of the same layer. Firstly, cosine similarity is used as the matrix similarity measure to measure the similarity between channels, guiding the network to prune. Secondly, the proposed method is, respectively, applied to filters and feature maps pruning, and the pruning effects in different layers are summarized. Finally, we propose a method to set the pruning rate dynamically according to the situation of each layer. Our method obtains extremely sparse networks with virtually the same accuracy as the reference networks on the CIFAR-10 and ImageNet ILSVRC-12 classification tasks. On CIFAR-10, our network achieves the 52.70% compression ratio on ResNet-56 and increases only 0.13% on top-1 error.
ISSN:1863-1703
1863-1711
DOI:10.1007/s11760-020-01760-x