The Belle II Raw Data Management System

The Belle II experiment, a major upgrade of the previous e + e − asymmetric collider experiment Belle, is expected to produce tens of petabytes of data per year due to the luminosity increase from the upgraded SuperKEKB accelerator. The distributed computing system of the Belle II experiment plays a...

Full description

Saved in:
Bibliographic Details
Published in:EPJ Web of Conferences 2020, Vol.245, p.4005
Main Authors: Hernández Villanueva, Michel, Ueda, Ikuo
Format: Article
Language:English
Subjects:
Computer networks
Data base management systems
Data management
Distributed processing
Experiments
Luminosity
Remote control
Storage
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c321t-a0f704974ed9c1b17646ac689d38078d6bf5573e3016fc592d5ae230b72424313
cites cdi_FETCH-LOGICAL-c321t-a0f704974ed9c1b17646ac689d38078d6bf5573e3016fc592d5ae230b72424313
container_end_page
container_issue
container_start_page 4005
container_title EPJ Web of Conferences
container_volume 245
creator Hernández Villanueva, Michel
Ueda, Ikuo
description The Belle II experiment, a major upgrade of the previous e + e − asymmetric collider experiment Belle, is expected to produce tens of petabytes of data per year due to the luminosity increase from the upgraded SuperKEKB accelerator. The distributed computing system of the Belle II experiment plays a key role, storing and distributing data in a reliable way to be easily accessed and analyzed by more than 1000 collaborators. In particular, the Belle II Raw Data Management system has been developed with an aim to upload output files onto grid storage, register them into the file and metadata catalogs, and make two replicas of the full raw data set using the Belle II Distributed Data Management system. It has been implemented as an extension of DIRAC (Distributed Infrastructure with Remote Agent Control) and consists of a database, services, client and monitoring tools, and several agents that treat the data automatically. The first year of data taken with the Belle II full detector has been managed by the Belle II Raw Data Management system successfully. The design, current status, and performance are presented. Prospects for improvements towards the full luminosity data taking are also reviewed.
doi_str_mv 10.1051/epjconf/202024504005
format article
fullrecord <record><control><sourceid>proquest_doaj_</sourceid><recordid>TN_cdi_doaj_primary_oai_doaj_org_article_753530deb2a74858aabe364891a2c756</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><doaj_id>oai_doaj_org_article_753530deb2a74858aabe364891a2c756</doaj_id><sourcerecordid>2465741833</sourcerecordid><originalsourceid>FETCH-LOGICAL-c321t-a0f704974ed9c1b17646ac689d38078d6bf5573e3016fc592d5ae230b72424313</originalsourceid><addsrcrecordid>eNpNkEtLw1AQhS-iYNH-AxcBF65i576TpdZXoCJoBXeXyc2kNqRJvUmR_nujLdKZxQzD4ZzhY-yCwzUHzSe0rnzblBMBQysNCkAfsZHgADFw9XF8sJ-ycddVMJRMU6nNiF3NPym6pbqmKMuiV_yO7rDH6BkbXNCKmj5623Y9rc7ZSYl1R-P9PGPvD_fz6VM8e3nMpjez2EvB-xihtKBSq6hIPc-5NcqgN0layARsUpi81NpKksBN6XUqCo0kJORWKKEkl2cs2_kWLVZuHZYrDFvX4tL9HdqwcBj6pa_JWS21hIJygVYlOkHMSRqVpByFt9oMXpc7r3VovzbU9a5qN6EZ3ndCGW0VT6QcVGqn8qHtukDlfyoH90vY7Qm7Q8LyB2dDa3w</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2465741833</pqid></control><display><type>article</type><title>The Belle II Raw Data Management System</title><source>Publicly Available Content Database (Proquest) (PQ_SDU_P3)</source><source>Free Full-Text Journals in Chemistry</source><creator>Hernández Villanueva, Michel ; Ueda, Ikuo</creator><contributor>Kamleh, W. ; Stewart, G.A. ; Kim, D. ; Doglioni, C. ; Silvestris, L. ; Jackson, P.</contributor><creatorcontrib>Hernández Villanueva, Michel ; Ueda, Ikuo ; Kamleh, W. ; Stewart, G.A. ; Kim, D. ; Doglioni, C. ; Silvestris, L. ; Jackson, P.</creatorcontrib><description>The Belle II experiment, a major upgrade of the previous e + e − asymmetric collider experiment Belle, is expected to produce tens of petabytes of data per year due to the luminosity increase from the upgraded SuperKEKB accelerator. The distributed computing system of the Belle II experiment plays a key role, storing and distributing data in a reliable way to be easily accessed and analyzed by more than 1000 collaborators. In particular, the Belle II Raw Data Management system has been developed with an aim to upload output files onto grid storage, register them into the file and metadata catalogs, and make two replicas of the full raw data set using the Belle II Distributed Data Management system. It has been implemented as an extension of DIRAC (Distributed Infrastructure with Remote Agent Control) and consists of a database, services, client and monitoring tools, and several agents that treat the data automatically. The first year of data taken with the Belle II full detector has been managed by the Belle II Raw Data Management system successfully. The design, current status, and performance are presented. Prospects for improvements towards the full luminosity data taking are also reviewed.</description><identifier>ISSN: 2100-014X</identifier><identifier>ISSN: 2101-6275</identifier><identifier>EISSN: 2100-014X</identifier><identifier>DOI: 10.1051/epjconf/202024504005</identifier><language>eng</language><publisher>Les Ulis: EDP Sciences</publisher><subject>Computer networks ; Data base management systems ; Data management ; Distributed processing ; Experiments ; Luminosity ; Remote control ; Storage</subject><ispartof>EPJ Web of Conferences, 2020, Vol.245, p.4005</ispartof><rights>2020. This work is licensed under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c321t-a0f704974ed9c1b17646ac689d38078d6bf5573e3016fc592d5ae230b72424313</citedby><cites>FETCH-LOGICAL-c321t-a0f704974ed9c1b17646ac689d38078d6bf5573e3016fc592d5ae230b72424313</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/2465741833?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>309,310,314,780,784,789,790,4024,23930,23931,25140,25753,27923,27924,27925,37012,44590</link.rule.ids></links><search><contributor>Kamleh, W.</contributor><contributor>Stewart, G.A.</contributor><contributor>Kim, D.</contributor><contributor>Doglioni, C.</contributor><contributor>Silvestris, L.</contributor><contributor>Jackson, P.</contributor><creatorcontrib>Hernández Villanueva, Michel</creatorcontrib><creatorcontrib>Ueda, Ikuo</creatorcontrib><title>The Belle II Raw Data Management System</title><title>EPJ Web of Conferences</title><description>The Belle II experiment, a major upgrade of the previous e + e − asymmetric collider experiment Belle, is expected to produce tens of petabytes of data per year due to the luminosity increase from the upgraded SuperKEKB accelerator. The distributed computing system of the Belle II experiment plays a key role, storing and distributing data in a reliable way to be easily accessed and analyzed by more than 1000 collaborators. In particular, the Belle II Raw Data Management system has been developed with an aim to upload output files onto grid storage, register them into the file and metadata catalogs, and make two replicas of the full raw data set using the Belle II Distributed Data Management system. It has been implemented as an extension of DIRAC (Distributed Infrastructure with Remote Agent Control) and consists of a database, services, client and monitoring tools, and several agents that treat the data automatically. The first year of data taken with the Belle II full detector has been managed by the Belle II Raw Data Management system successfully. The design, current status, and performance are presented. Prospects for improvements towards the full luminosity data taking are also reviewed.</description><subject>Computer networks</subject><subject>Data base management systems</subject><subject>Data management</subject><subject>Distributed processing</subject><subject>Experiments</subject><subject>Luminosity</subject><subject>Remote control</subject><subject>Storage</subject><issn>2100-014X</issn><issn>2101-6275</issn><issn>2100-014X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><sourceid>DOA</sourceid><recordid>eNpNkEtLw1AQhS-iYNH-AxcBF65i576TpdZXoCJoBXeXyc2kNqRJvUmR_nujLdKZxQzD4ZzhY-yCwzUHzSe0rnzblBMBQysNCkAfsZHgADFw9XF8sJ-ycddVMJRMU6nNiF3NPym6pbqmKMuiV_yO7rDH6BkbXNCKmj5623Y9rc7ZSYl1R-P9PGPvD_fz6VM8e3nMpjez2EvB-xihtKBSq6hIPc-5NcqgN0layARsUpi81NpKksBN6XUqCo0kJORWKKEkl2cs2_kWLVZuHZYrDFvX4tL9HdqwcBj6pa_JWS21hIJygVYlOkHMSRqVpByFt9oMXpc7r3VovzbU9a5qN6EZ3ndCGW0VT6QcVGqn8qHtukDlfyoH90vY7Qm7Q8LyB2dDa3w</recordid><startdate>2020</startdate><enddate>2020</enddate><creator>Hernández Villanueva, Michel</creator><creator>Ueda, Ikuo</creator><general>EDP Sciences</general><scope>AAYXX</scope><scope>CITATION</scope><scope>8FE</scope><scope>8FG</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>P5Z</scope><scope>P62</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>DOA</scope></search><sort><creationdate>2020</creationdate><title>The Belle II Raw Data Management System</title><author>Hernández Villanueva, Michel ; Ueda, Ikuo</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c321t-a0f704974ed9c1b17646ac689d38078d6bf5573e3016fc592d5ae230b72424313</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Computer networks</topic><topic>Data base management systems</topic><topic>Data management</topic><topic>Distributed processing</topic><topic>Experiments</topic><topic>Luminosity</topic><topic>Remote control</topic><topic>Storage</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Hernández Villanueva, Michel</creatorcontrib><creatorcontrib>Ueda, Ikuo</creatorcontrib><collection>CrossRef</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>Advanced Technologies &amp; Aerospace Database‎ (1962 - current)</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>SciTech Premium Collection</collection><collection>Advanced Technologies &amp; Aerospace Database</collection><collection>ProQuest Advanced Technologies &amp; Aerospace Collection</collection><collection>Publicly Available Content Database (Proquest) (PQ_SDU_P3)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Directory of Open Access Journals</collection><jtitle>EPJ Web of Conferences</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Hernández Villanueva, Michel</au><au>Ueda, Ikuo</au><au>Kamleh, W.</au><au>Stewart, G.A.</au><au>Kim, D.</au><au>Doglioni, C.</au><au>Silvestris, L.</au><au>Jackson, P.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>The Belle II Raw Data Management System</atitle><jtitle>EPJ Web of Conferences</jtitle><date>2020</date><risdate>2020</risdate><volume>245</volume><spage>4005</spage><pages>4005-</pages><issn>2100-014X</issn><issn>2101-6275</issn><eissn>2100-014X</eissn><abstract>The Belle II experiment, a major upgrade of the previous e + e − asymmetric collider experiment Belle, is expected to produce tens of petabytes of data per year due to the luminosity increase from the upgraded SuperKEKB accelerator. The distributed computing system of the Belle II experiment plays a key role, storing and distributing data in a reliable way to be easily accessed and analyzed by more than 1000 collaborators. In particular, the Belle II Raw Data Management system has been developed with an aim to upload output files onto grid storage, register them into the file and metadata catalogs, and make two replicas of the full raw data set using the Belle II Distributed Data Management system. It has been implemented as an extension of DIRAC (Distributed Infrastructure with Remote Agent Control) and consists of a database, services, client and monitoring tools, and several agents that treat the data automatically. The first year of data taken with the Belle II full detector has been managed by the Belle II Raw Data Management system successfully. The design, current status, and performance are presented. Prospects for improvements towards the full luminosity data taking are also reviewed.</abstract><cop>Les Ulis</cop><pub>EDP Sciences</pub><doi>10.1051/epjconf/202024504005</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 2100-014X
ispartof EPJ Web of Conferences, 2020, Vol.245, p.4005
issn 2100-014X
2101-6275
2100-014X
language eng
recordid cdi_doaj_primary_oai_doaj_org_article_753530deb2a74858aabe364891a2c756
source Publicly Available Content Database (Proquest) (PQ_SDU_P3); Free Full-Text Journals in Chemistry
subjects Computer networks
Data base management systems
Data management
Distributed processing
Experiments
Luminosity
Remote control
Storage
title The Belle II Raw Data Management System
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-03T11%3A24%3A44IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_doaj_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=The%20Belle%20II%20Raw%20Data%20Management%20System&rft.jtitle=EPJ%20Web%20of%20Conferences&rft.au=Hern%C3%A1ndez%20Villanueva,%20Michel&rft.date=2020&rft.volume=245&rft.spage=4005&rft.pages=4005-&rft.issn=2100-014X&rft.eissn=2100-014X&rft_id=info:doi/10.1051/epjconf/202024504005&rft_dat=%3Cproquest_doaj_%3E2465741833%3C/proquest_doaj_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c321t-a0f704974ed9c1b17646ac689d38078d6bf5573e3016fc592d5ae230b72424313%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2465741833&rft_id=info:pmid/&rfr_iscdi=true