Smart Memory: Deep Learning Acceleration in 3D-Stacked Memories

Processing-in-memory (PIM) is the most promising paradigm to address the bandwidth bottleneck in deep neural network (DNN) accelerators. However, the algorithmic and dataflow structure of DNNs still necessitates moving a large amount of data across banks inside the memory device to bring input data...

Full description

Saved in:
Bibliographic Details
Published in:IEEE computer architecture letters 2024-01, Vol.23 (1), p.137-141
Main Authors: Rezaei, Seyyed Hossein SeyyedAghaei, Moghaddam, Parham Zilouchian, Modarressi, Mehdi
Format: Article
Language:English
Subjects:
3D-stacked memory
Accelerators
Artificial neural networks
Bandwidth
Bandwidths
Bottlenecks
Computer architecture
Data communication
Data transmission
deep learning accelerator
Distributed memory
Machine learning
Memory devices
Memory management
Network-on-memory
Neural networks
processing-in-memory
Random access memory
Switches
System on chip
Three-dimensional displays
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Processing-in-memory (PIM) is the most promising paradigm to address the bandwidth bottleneck in deep neural network (DNN) accelerators. However, the algorithmic and dataflow structure of DNNs still necessitates moving a large amount of data across banks inside the memory device to bring input data and their corresponding model parameters together, negatively shifting part of the bandwidth bottleneck to the in-memory data communication infrastructure. To alleviate this bottleneck, we present Smart Memory , a highly parallel in-memory DNN accelerator for 3D memories that benefits from a scalable high-bandwidth in-memory network. Whereas the existing PIM designs implement the compute units and network-on-chip on the logic die of the underlying 3D memory, in Smart Memory the computation and data transmission tasks are distributed across the memory banks. To this end, each memory bank is equipped with (1) a very simple processing unit to run neural networks, and (2) a circuit-switched router to interconnect memory banks by a 3D network-on-memory. Our evaluation shows 44% average performance improvement over state-of-the-art in-memory DNN accelerators.
ISSN:1556-6056
1556-6064
DOI:10.1109/LCA.2023.3287976