DyNNamic: Dynamically Reshaping, High Data-Reuse Accelerator for Compact DNNs

Convolutional layers dominate the computation and energy costs of Deep Neural Network (DNN) inference. Recent algorithmic works attempt to reduce these bottlenecks via compact DNN structures and model compression. Likewise, state-of-the-art accelerator designs leverage spatiotemporal characteristics...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on computers 2023-03, Vol.72 (3), p.880-892
Main Authors: Hanson, Edward, Li, Shiyu, Qian, Xuehai, Li, Hai Helen, Chen, Yiran
Format: Article
Language:English
Subjects:
adaptable architectures
Algorithms
Arrays
Artificial neural networks
Computer architecture
Convolution
Costs
Dataflow architectures
Energy costs
Hardware
Inference
Kernel
Kernels
machine learning
Matrix decomposition
Network latency
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Convolutional layers dominate the computation and energy costs of Deep Neural Network (DNN) inference. Recent algorithmic works attempt to reduce these bottlenecks via compact DNN structures and model compression. Likewise, state-of-the-art accelerator designs leverage spatiotemporal characteristics of convolutional layers to reduce data movement overhead and improve throughput. Although both are independently effective at reducing latency and energy costs, combining these approaches does not guarantee cumulative improvements due to inefficient mapping. This inefficiency can be attributed to (1) inflexibility of underlying hardware and (2) inherent reduction of data-reuse opportunities of compact DNN structures. To address these issues, we propose a dynamically reshaping, high data-reuse PE array accelerator, namely DyNNamic . DyNNamic leverages kernel-wise filter decomposition to partition the convolution operation into two compact stages: Shared Kernels Convolution (SKC) and Weighted Accumulation (WA). Because both stages have vastly different dimensions, DyNNamic reshapes its PE array to effectively map the algorithm to the architecture. The architecture then exploits data-reuse opportunities created by the SKC stage, further reducing data movement with negligible overhead. We evaluate our approach on various representative networks and compare against state-of-the-art accelerators. On average, DyNNamic outperforms DianNao by 8.4\times 8 . 4 × and 12.3\times 12 . 3 × in terms of inference energy and latency, respectively.
ISSN:0018-9340
1557-9956
DOI:10.1109/TC.2022.3184272