Practical off-chip meta-data for temporal memory streaming

• Main Authors: Wenisch, T.F., Ferdman, M., Ailamaki, A., Falsafi, B., Moshovos, A.
• Format: Conference Proceeding
• Language: English
• Subjects: Bandwidth; Costs; Delay; Hardware; Organizing; Prefetching; Sampling methods; Table lookup; Throughput; Web server
• ISSN: 1530-0897; 2378-203X
• DOI: 10.1109/HPCA.2009.4798239

Prior research demonstrates that temporal memory streaming and related address-correlating prefetchers improve performance of commercial server workloads though increased memory level parallelism. Unfortunately, these prefetchers require large on-chip meta-data storage, making previously-proposed designs impractical. Hence, to improve practicality, researchers have sought ways to enable timely prefetch while locating meta-data entirely off-chip. Unfortunately, current solutions for off-chip meta-data increase memory traffic by over a factor of three. We observe three requirements to store meta-data off chip: minimal off-chip lookup latency, bandwidth-efficient meta-data updates, and off-chip lookup amortized over many prefetches. In this work, we show: (1) minimal off-chip meta-data lookup latency can be achieved through a hardware-managed main memory hash table, (2) bandwidth-efficient updates can be performed through probabilistic sampling of meta-data updates, and (3) off-chip lookup costs can be amortized by organizing meta-data to allow a single lookup to yield long prefetch sequences. Using these techniques, we develop sampled temporal memory streaming (STMS), a practical address-correlating prefetcher that keeps predictor meta-data in main memory while achieving 90% of the performance potential of idealized on-chip meta-data storage.