Operating system benchmarking in the wake of lmbench: a case study of the performance of NetBSD on the Intel x86 architecture

• Main Authors: Brown, Aaron B., Seltzer, Margo I.
• Format: Conference Proceeding
• Language: English
• Subjects: Computer systems organization > Dependable and fault-tolerant systems and networks; General and reference > Cross-computing tools and techniques > Evaluation; General and reference > Cross-computing tools and techniques > Metrics; General and reference > Cross-computing tools and techniques > Performance; Networks > Network performance evaluation; Networks > Network protocols; Software and its engineering > Software organization and properties > Contextual software domains > Operating systems; Theory of computation > Models of computation > Probabilistic computation
• ISSN: 0163-5999
• DOI: 10.1145/258612.258690

The lmbench suite of operating system microbenchmarks provides a set of portable programs for use in cross-platform comparisons. We have augmented the lmbench suite to increase its flexibility and precision, and to improve its methodological and statistical operation. This enables the detailed study of interactions between the operating system and the hardware architecture. We describe modifications to lmbench, and then use our new benchmark suite, hbench:OS, to examine how the performance of operating system primitives under NetBSD has scaled with the processor evolution of the Intel x86 architecture. Our analysis shows that off-chip memory system design continues to influence operating system performance in a significant way and that key design decisions (such as suboptimal choices of DRAM and cache technology, and memory-bus and cache coherency protocols) can essentially nullify the performance benefits of the aggressive execution core and sophisticated on-chip memory system of a modern processor such as the Intel Pentium Pro.