Using server-to-server communication in parallel file systems to simplify consistency and improve performance

• Main Authors: Carns, Philip H., Settlemyer, Bradley W., Ligon, Walter B.
• Format: Conference Proceeding
• Language: English
• Subjects: Computer systems organization > Dependable and fault-tolerant systems and networks; General and reference > Cross-computing tools and techniques > Performance; Information systems > Information storage systems > Record storage systems; Networks > Network performance evaluation; Software and its engineering > Software organization and properties > Contextual software domains > Operating systems > File systems management; Theory of computation > Design and analysis of algorithms > Data structures design and analysis > Sorting and searching
• DOI: 10.5555/1413370.1413377

The trend in parallel computing toward clusters running thousands of cooperating processes per application has led to an I/O bottleneck that has only gotten more severe as the CPU density of clusters has increased. Current parallel file systems provide large amounts of aggregate I/O bandwidth; however, they do not achieve the high degrees of metadata scalability required to manage files distributed across hundreds or thousands of storage nodes. In this paper we examine the use of collective communication between the storage servers to improve the scalability of file metadata operations. In particular, we apply server-to-server communication to simplify consistency checking and improve the performance of file creation, file removal, and file stat. Our results indicate that collective communication is an effective scheme for simplifying consistency checks and significantly improving the performance for several real metadata intensive workloads.