Smart Memory: Deep Learning Acceleration in 3D-Stacked Memories

• Published in: IEEE computer architecture letters 2024-01, Vol.23 (1), p.137-141
• Main Authors: Rezaei, Seyyed Hossein SeyyedAghaei, Moghaddam, Parham Zilouchian, Modarressi, Mehdi
• Format: Article
• Language: English
• Subjects: 3D-stacked memory; Accelerators; Artificial neural networks; Bandwidth; Bandwidths; Bottlenecks; Computer architecture; Data communication; Data transmission; deep learning accelerator; Distributed memory; Machine learning; Memory devices; Memory management; Network-on-memory; Neural networks; processing-in-memory; Random access memory; Switches; System on chip; Three-dimensional displays
• ISSN: 1556-6056; 1556-6064
• DOI: 10.1109/LCA.2023.3287976

Processing-in-memory (PIM) is the most promising paradigm to address the bandwidth bottleneck in deep neural network (DNN) accelerators. However, the algorithmic and dataflow structure of DNNs still necessitates moving a large amount of data across banks inside the memory device to bring input data and their corresponding model parameters together, negatively shifting part of the bandwidth bottleneck to the in-memory data communication infrastructure. To alleviate this bottleneck, we present Smart Memory , a highly parallel in-memory DNN accelerator for 3D memories that benefits from a scalable high-bandwidth in-memory network. Whereas the existing PIM designs implement the compute units and network-on-chip on the logic die of the underlying 3D memory, in Smart Memory the computation and data transmission tasks are distributed across the memory banks. To this end, each memory bank is equipped with (1) a very simple processing unit to run neural networks, and (2) a circuit-switched router to interconnect memory banks by a 3D network-on-memory. Our evaluation shows 44% average performance improvement over state-of-the-art in-memory DNN accelerators.