Remote data access in computational jobs on the ATLAS data grid

This work describes the technique of remote data access from computational jobs on the ATLAS data grid. In comparison to traditional data movement and stage-in approaches it is well suited for data transfers which are asynchronous with respect to the job execution. Hence, it can be used for optimiza...

Full description

Saved in:
Bibliographic Details
Published in:EPJ Web of conferences 2019-01, Vol.214, p.4028
Main Authors: Begy, Volodimir, Barisits, Martin, Lassnig, Mario, Schikuta, Erich
Format: Article
Language:English
Subjects:
Optimization
Parameter identification
Storage area networks
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites cdi_FETCH-LOGICAL-c306t-2c6d2e27fb48c31bfc5dc50cb982287600d2b93d8f720258fb9630af517a42583
container_end_page
container_issue
container_start_page 4028
container_title EPJ Web of conferences
container_volume 214
creator Begy, Volodimir
Barisits, Martin
Lassnig, Mario
Schikuta, Erich
description This work describes the technique of remote data access from computational jobs on the ATLAS data grid. In comparison to traditional data movement and stage-in approaches it is well suited for data transfers which are asynchronous with respect to the job execution. Hence, it can be used for optimization of data access patterns based on various policies. In this study, remote data access is realized with the HTTP and WebDAV protocols, and is investigated in the context of intra- and inter-computing site data transfers. In both cases, the typical scenarios for application of remote data access are identified. The paper also presents an analysis of parameters influencing the data goodput between heterogeneous storage element - worker node pairs on the grid.
doi_str_mv 10.1051/epjconf/201921404028
format article
fullrecord <record><control><sourceid>proquest_doaj_</sourceid><recordid>TN_cdi_doaj_primary_oai_doaj_org_article_bccb71ae36bc4d5ab27bf85a1ee7c5c8</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><doaj_id>oai_doaj_org_article_bccb71ae36bc4d5ab27bf85a1ee7c5c8</doaj_id><sourcerecordid>2297142057</sourcerecordid><originalsourceid>FETCH-LOGICAL-c306t-2c6d2e27fb48c31bfc5dc50cb982287600d2b93d8f720258fb9630af517a42583</originalsourceid><addsrcrecordid>eNpNkF1LwzAUhosoOOb-gRcFr-tOkqZpr2QMPwYDQSd4F5LTZLZsTU2yC_-9nR2yc3M-eHhfzpsktwTuCXAyN32LrrNzCqSiJIccaHmRTCgByIDkn5dn83UyC6GFoVhVMV5Mkoc3s3fRpLWKKlWIJoS06VJ0-_4QVWxcp3Zp63RIXZfGL5MuNuvF-4hvfVPfJFdW7YKZnfo0-Xh63CxfsvXr82q5WGfIoIgZxaKmhgqr8xIZ0RZ5jRxQVyWlpSgAaqorVpdWUKC8tLoqGCjLiVD5sLNpshp1a6da2ftmr_yPdKqRfwfnt1L52ODOSI2oBVGGFRrzmitNhbYlV8QYgRyPWnejVu_d98GEKFt38MOjQVJaCZJT4GKg8pFC70Lwxv67EpDH5OUpeXmePPsFnB52uA</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2297142057</pqid></control><display><type>article</type><title>Remote data access in computational jobs on the ATLAS data grid</title><source>Publicly Available Content Database</source><source>Free Full-Text Journals in Chemistry</source><creator>Begy, Volodimir ; Barisits, Martin ; Lassnig, Mario ; Schikuta, Erich</creator><contributor>Hristov, P. ; Smirnova, O. ; Betev, L. ; Forti, A. ; Litmaath, M.</contributor><creatorcontrib>Begy, Volodimir ; Barisits, Martin ; Lassnig, Mario ; Schikuta, Erich ; Hristov, P. ; Smirnova, O. ; Betev, L. ; Forti, A. ; Litmaath, M.</creatorcontrib><description>This work describes the technique of remote data access from computational jobs on the ATLAS data grid. In comparison to traditional data movement and stage-in approaches it is well suited for data transfers which are asynchronous with respect to the job execution. Hence, it can be used for optimization of data access patterns based on various policies. In this study, remote data access is realized with the HTTP and WebDAV protocols, and is investigated in the context of intra- and inter-computing site data transfers. In both cases, the typical scenarios for application of remote data access are identified. The paper also presents an analysis of parameters influencing the data goodput between heterogeneous storage element - worker node pairs on the grid.</description><identifier>ISSN: 2100-014X</identifier><identifier>ISSN: 2101-6275</identifier><identifier>EISSN: 2100-014X</identifier><identifier>DOI: 10.1051/epjconf/201921404028</identifier><language>eng</language><publisher>Les Ulis: EDP Sciences</publisher><subject>Optimization ; Parameter identification ; Storage area networks</subject><ispartof>EPJ Web of conferences, 2019-01, Vol.214, p.4028</ispartof><rights>2019. This work is licensed under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c306t-2c6d2e27fb48c31bfc5dc50cb982287600d2b93d8f720258fb9630af517a42583</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/2297142057?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>309,310,314,780,784,789,790,23929,23930,25139,25752,27923,27924,37011,44589</link.rule.ids></links><search><contributor>Hristov, P.</contributor><contributor>Smirnova, O.</contributor><contributor>Betev, L.</contributor><contributor>Forti, A.</contributor><contributor>Litmaath, M.</contributor><creatorcontrib>Begy, Volodimir</creatorcontrib><creatorcontrib>Barisits, Martin</creatorcontrib><creatorcontrib>Lassnig, Mario</creatorcontrib><creatorcontrib>Schikuta, Erich</creatorcontrib><title>Remote data access in computational jobs on the ATLAS data grid</title><title>EPJ Web of conferences</title><description>This work describes the technique of remote data access from computational jobs on the ATLAS data grid. In comparison to traditional data movement and stage-in approaches it is well suited for data transfers which are asynchronous with respect to the job execution. Hence, it can be used for optimization of data access patterns based on various policies. In this study, remote data access is realized with the HTTP and WebDAV protocols, and is investigated in the context of intra- and inter-computing site data transfers. In both cases, the typical scenarios for application of remote data access are identified. The paper also presents an analysis of parameters influencing the data goodput between heterogeneous storage element - worker node pairs on the grid.</description><subject>Optimization</subject><subject>Parameter identification</subject><subject>Storage area networks</subject><issn>2100-014X</issn><issn>2101-6275</issn><issn>2100-014X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><sourceid>DOA</sourceid><recordid>eNpNkF1LwzAUhosoOOb-gRcFr-tOkqZpr2QMPwYDQSd4F5LTZLZsTU2yC_-9nR2yc3M-eHhfzpsktwTuCXAyN32LrrNzCqSiJIccaHmRTCgByIDkn5dn83UyC6GFoVhVMV5Mkoc3s3fRpLWKKlWIJoS06VJ0-_4QVWxcp3Zp63RIXZfGL5MuNuvF-4hvfVPfJFdW7YKZnfo0-Xh63CxfsvXr82q5WGfIoIgZxaKmhgqr8xIZ0RZ5jRxQVyWlpSgAaqorVpdWUKC8tLoqGCjLiVD5sLNpshp1a6da2ftmr_yPdKqRfwfnt1L52ODOSI2oBVGGFRrzmitNhbYlV8QYgRyPWnejVu_d98GEKFt38MOjQVJaCZJT4GKg8pFC70Lwxv67EpDH5OUpeXmePPsFnB52uA</recordid><startdate>20190101</startdate><enddate>20190101</enddate><creator>Begy, Volodimir</creator><creator>Barisits, Martin</creator><creator>Lassnig, Mario</creator><creator>Schikuta, Erich</creator><general>EDP Sciences</general><scope>AAYXX</scope><scope>CITATION</scope><scope>8FE</scope><scope>8FG</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>P5Z</scope><scope>P62</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>DOA</scope></search><sort><creationdate>20190101</creationdate><title>Remote data access in computational jobs on the ATLAS data grid</title><author>Begy, Volodimir ; Barisits, Martin ; Lassnig, Mario ; Schikuta, Erich</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c306t-2c6d2e27fb48c31bfc5dc50cb982287600d2b93d8f720258fb9630af517a42583</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Optimization</topic><topic>Parameter identification</topic><topic>Storage area networks</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Begy, Volodimir</creatorcontrib><creatorcontrib>Barisits, Martin</creatorcontrib><creatorcontrib>Lassnig, Mario</creatorcontrib><creatorcontrib>Schikuta, Erich</creatorcontrib><collection>CrossRef</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>Advanced Technologies &amp; Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>AUTh Library subscriptions: ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>Advanced Technologies &amp; Aerospace Database</collection><collection>ProQuest Advanced Technologies &amp; Aerospace Collection</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>EPJ Web of conferences</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Begy, Volodimir</au><au>Barisits, Martin</au><au>Lassnig, Mario</au><au>Schikuta, Erich</au><au>Hristov, P.</au><au>Smirnova, O.</au><au>Betev, L.</au><au>Forti, A.</au><au>Litmaath, M.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Remote data access in computational jobs on the ATLAS data grid</atitle><jtitle>EPJ Web of conferences</jtitle><date>2019-01-01</date><risdate>2019</risdate><volume>214</volume><spage>4028</spage><pages>4028-</pages><issn>2100-014X</issn><issn>2101-6275</issn><eissn>2100-014X</eissn><abstract>This work describes the technique of remote data access from computational jobs on the ATLAS data grid. In comparison to traditional data movement and stage-in approaches it is well suited for data transfers which are asynchronous with respect to the job execution. Hence, it can be used for optimization of data access patterns based on various policies. In this study, remote data access is realized with the HTTP and WebDAV protocols, and is investigated in the context of intra- and inter-computing site data transfers. In both cases, the typical scenarios for application of remote data access are identified. The paper also presents an analysis of parameters influencing the data goodput between heterogeneous storage element - worker node pairs on the grid.</abstract><cop>Les Ulis</cop><pub>EDP Sciences</pub><doi>10.1051/epjconf/201921404028</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 2100-014X
ispartof EPJ Web of conferences, 2019-01, Vol.214, p.4028
issn 2100-014X
2101-6275
2100-014X
language eng
recordid cdi_doaj_primary_oai_doaj_org_article_bccb71ae36bc4d5ab27bf85a1ee7c5c8
source Publicly Available Content Database; Free Full-Text Journals in Chemistry
subjects Optimization
Parameter identification
Storage area networks
title Remote data access in computational jobs on the ATLAS data grid
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-11T18%3A00%3A56IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_doaj_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Remote%20data%20access%20in%20computational%20jobs%20on%20the%20ATLAS%20data%20grid&rft.jtitle=EPJ%20Web%20of%20conferences&rft.au=Begy,%20Volodimir&rft.date=2019-01-01&rft.volume=214&rft.spage=4028&rft.pages=4028-&rft.issn=2100-014X&rft.eissn=2100-014X&rft_id=info:doi/10.1051/epjconf/201921404028&rft_dat=%3Cproquest_doaj_%3E2297142057%3C/proquest_doaj_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c306t-2c6d2e27fb48c31bfc5dc50cb982287600d2b93d8f720258fb9630af517a42583%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2297142057&rft_id=info:pmid/&rfr_iscdi=true