An Efficient Memristor-Based Circuit Implementation of Squeeze-and-Excitation Fully Convolutional Neural Networks

Recently, there has been a surge of interest in applying memristors to hardware implementations of deep neural networks due to various desirable properties of the memristor, such as nonvolativity, multivalue, and nanosize. Most existing neural network circuit designs, however, are based on generic f...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2022-04, Vol.33 (4), p.1779-1790
Main Authors: Chen, Jiadong, Wu, Yincheng, Yang, Yin, Wen, Shiping, Shi, Kaibo, Bermak, Amine, Huang, Tingwen
Format: Article
Language:English
Subjects:
Artificial neural networks
Circuit design
Circuit synthesis
Convolution
Excitation
Fully convolutional network
Hardware
Image analysis
Image processing
Image segmentation
Kernel
Medical imaging
memristor circuit
Memristors
Neural networks
Power consumption
Pruning
Reduction
squeeze-and-excitation (SE)
weight pruning
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recently, there has been a surge of interest in applying memristors to hardware implementations of deep neural networks due to various desirable properties of the memristor, such as nonvolativity, multivalue, and nanosize. Most existing neural network circuit designs, however, are based on generic frameworks that are not optimized for memristors. Furthermore, to the best of our knowledge, there are no existing efficient memristor-based implementations of complex neural network operators, such as deconvolutions and squeeze-and-excitation (SE) blocks, which are critical for achieving high accuracy in common medical image analysis applications, such as semantic segmentation. This article proposes convolution-kernel first (CKF), an efficient scheme for designing memristor-based fully convolutional neural networks (FCNs). Compared with existing neural network circuits, CKF enables effective parameter pruning, which significantly reduces circuit power consumption. Furthermore, CKF includes the novel, memristor-optimized implementations of deconvolution layers and SE blocks. Simulation results on real medical image segmentation tasks confirm that CKF obtains up to 56.2% reduction in terms of computations and 33.62-W reduction in terms of power consumption in the circuit after weight pruning while retaining high accuracy on the test set. Moreover, the pruning results can be applied directly to existing circuits without any modification for the corresponding system.
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2020.3044047