Using RDMA efficiently for key-value services

• Published in: Computer communication review 2014-10, Vol.44 (4), p.295-306
• Main Authors: Kalia, Anuj, Kaminsky, Michael, Andersen, David G.
• Format: Magazinearticle
• Language: English
• Subjects: Bypasses; Central processing units; Client server systems; Design engineering; Hardware; Networks; Servers; Servers (computers)
• ISSN: 0146-4833
• DOI: 10.1145/2740070.2626299

This paper describes the design and implementation of HERD, a key-value system designed to make the best use of an RDMA network. Unlike prior RDMA-based key-value systems, HERD focuses its design on reducing network round trips while using efficient RDMA primitives; the result is substantially lower latency, and throughput that saturates modern, commodity RDMA hardware. HERD has two unconventional decisions: First, it does not use RDMA reads, despite the allure of operations that bypass the remote CPU entirely. Second, it uses a mix of RDMA and messaging verbs, despite the conventional wisdom that the messaging primitives are slow. A HERD client writes its request into the server's memory; the server computes the reply. This design uses a single round trip for all requests and supports up to 26 million key-value operations per second with 5μs average latency. Notably, for small key-value items, our full system throughput is similar to native RDMA read throughput and is over 2X higher than recent RDMA-based key-value systems. We believe that HERD further serves as an effective template for the construction of RDMA-based datacenter services.