NoM: Network-on-Memory for Inter-Bank Data Transfer in Highly-Banked Memories

• Published in: IEEE computer architecture letters 2020-01, Vol.19 (1), p.80-83
• Main Authors: Rezaei, Seyyed Hossein SeyyedAghaei, Modarressi, Mehdi, Ausavarungnirun, Rachata, Sadrosadati, Mohammad, Mutlu, Onur, Daneshtalab, Masoud
• Format: Article
• Language: English
• Subjects: 3D-Stacked Memory; Circuit design; Circuit Switching; Communication schemes; Conventional computers; Conventional techniques; Data communication; Data communication systems; Data Copy; Data transfer; Data transfer (computers); Data-intensive workloads; Dynamic random access memory; Dynamic random access storage; Memory controller; Memory Network; Memory operations; Memory Systems; Microprocessors; Performance enhancement; State-of-the-art approach; State-of-the-art techniques; Three dimensional integrated circuits
• ISSN: 1556-6056; 1556-6064
• DOI: 10.1109/LCA.2020.2990599

Data copy is a widely-used memory operation in many programs and operating system services. In conventional computers, data copy is often carried out by two separate read and write transactions that pass data back and forth between the DRAM chip and the processor chip. Some prior mechanisms propose to avoid this unnecessary data movement by using the shared internal bus in the DRAM chip to directly copy data within the DRAM chip (e.g., between two DRAM banks). While these methods exhibit superior performance compared to conventional techniques, data copy across different DRAM banks is still greatly slower than data copy within the same DRAM bank. Hence, these techniques have limited benefit for the emerging 3D-stacked memories (e.g., HMC and HBM) that contain hundreds of DRAM banks across multiple memory controllers. In this paper, we present Network-on-Memory (NoM), a lightweight inter-bank data communication scheme that enables direct data copy across both memory banks of a 3D-stacked memory. NoM adopts a TDM-based circuit-switching design, where circuit setup is done by the memory controller. Compared to state-of-the-art approaches, NoM enables both fast data copy between multiple DRAM banks and concurrent data transfer operations. Our evaluation shows that NoM improves the performance of data-intensive workloads by 3.8X and 75 percent, on average, compared to the baseline conventional 3D-stacked DRAM architecture and state-of-the-art techniques, respectively.