New approach to global data access in computational infrastructures

• Published in: Future generation computer systems 2021-12, Vol.125, p.575-589
• Main Authors: Wrzeszcz, Michał, Dutka, Łukasz, Słota, Renata G., Kitowski, Jacek
• Format: Article
• Language: English
• Subjects: Clouds; Data access system; Data-intensive computing; Decentralized systems; Heterogeneous storage systems; Open data; Virtual file system
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2021.06.054

Data access and management on a large, global scale is currently at the center of scientific interest. This follows from the need for data access from multi- and hybrid-cloud applications. In most cases existing solutions provide sufficient functionality to scale computing resources but scaling resources in terms of efficient data access e.g. for data-intensive applications is still not comprehensively resolved. In this paper, we present a new approach to global data access that supports the execution of data-intensive applications on globally distributed heterogeneous resources provided and managed by independent providers. We identify the functionality, representation, and processing of contextual information in the form of metadata, and the organization of data, resulting in four models describing the details of the approach. An experimental evaluation of this approach is discussed. We show overheads for single- and multi-site environments, system scalability, and usage of context awareness to achieve desired behavior proving insignificant overhead introduced by the system. The results confirm usability of the approach to supporting computations in heterogeneous environments with unified access to data distributed worldwide achieved by using broad contextual information. •Unified data access at any time and place, despite of data location and data providers.•Contextual information to automatic adjustment of access to data distributed worldwide.•A reference implementation of Onedata system to illustrate capabilities of the approach.•Evaluation of performance and context awareness to show benefits of the approach.