A Non-Forced-Write Atomic Commit Protocol for Cluster File Systems

Distributed metadata consistency is one of the critical issues of metadata clusters in distributed file systems. Existing methods to maintain metadata consistency generally need several log forced write operations. Since synchronous disk IO is very inefficient, the average response time of metadata...

Full description

Saved in:
Bibliographic Details
Published in:Journal of computer science and technology 2014, Vol.29 (2), p.303-315
Main Author: 邵冰清 张军伟 郑彩平 张浩 刘振军 许鲁
Format: Article
Language:English
Subjects:
Artificial Intelligence
Clusters
Computer Science
Consistency
Data Structures and Information Theory
Disks
Distributed processing
Information Systems Applications (incl.Internet)
Metadata
Networks
Protocol
Protocol (computers)
R&D
Regular Paper
Research & development
Response time
Science
Servers
Servers (computers)
Software Engineering
Studies
Theory of Computation
元数据服务器
分布式文件系统
原子
平均响应时间
提交协议
数据一致性
数据操作
群集
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites cdi_FETCH-LOGICAL-c364t-a14a965e7c3bda6335757472c17b075ba06f62f939ee96dde6f3fbe8f72199aa3
container_end_page 315
container_issue 2
container_start_page 303
container_title Journal of computer science and technology
container_volume 29
creator 邵冰清 张军伟 郑彩平 张浩 刘振军 许鲁
description Distributed metadata consistency is one of the critical issues of metadata clusters in distributed file systems. Existing methods to maintain metadata consistency generally need several log forced write operations. Since synchronous disk IO is very inefficient, the average response time of metadata operations is greatly increased. In this paper, an asynchronous atomic commit protocol （ACP） named Dual-Log （DL） is presented. It does not need any log forced write operations. Optimizing for distributed metadata operations involving only two metadata servers, DL mutually records the redo log in counterpart metadata servers by transferring through the low latency network. A crashed metadata server can redo the metadata operation with the redundant redo log. Since the latency of the network is much lower than the latency of disk IO, DL can improve the performance of distributed metadata service significantly. The prototype of DL is implemented based on local journal. The performance is tested by comparing with two widely used protocols, EP and S2PC-MP, and the results show that the average response time of distributed metadata operations is reduced by about 40%-60%, and the recovery time is only I second under 10 thousands uncompleted distributed metadata operations.
doi_str_mv 10.1007/s11390-014-1432-y
format article
fullrecord <record><control><sourceid>wanfang_jour_proqu</sourceid><recordid>TN_cdi_wanfang_journals_jsjkxjsxb_e201402013</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><cqvip_id>48941609</cqvip_id><wanfj_id>jsjkxjsxb_e201402013</wanfj_id><sourcerecordid>jsjkxjsxb_e201402013</sourcerecordid><originalsourceid>FETCH-LOGICAL-c364t-a14a965e7c3bda6335757472c17b075ba06f62f939ee96dde6f3fbe8f72199aa3</originalsourceid><addsrcrecordid>eNp9kTFPwzAQhSMEEqXwA9iCWBgwnGPHrscSUUCqAAkQo-WkdklI4tZORfvvcRVUIQaW8w3fe-_kF0WnGK4wAL_2GBMBCDBFmJIEbfaiAR4xQJRTsR92AEAijMPoyPsKgHCgdBDdjONH26KJdYWeoXdXdjoed7YpizizTVN28bOznS1sHRvr4qxe-U67eFLWOn7ZhL3xx9GBUbXXJz_vMHqb3L5m92j6dPeQjaeoIIx2SGGqBEs1L0g-U4yQlKec8qTAPAee5gqYYYkRRGgt2GymmSEm1yPDEyyEUmQYXfa-X6o1qp3Lyq5cGxJl5avPdeXXudRJ-AEIgwT8oscXzi5X2neyKX2h61q12q68xGkCYkQSTgN6_gfdWeOUUExFCiJQuKcKZ7132siFKxvlNhKD3JYg-xJkOEFuS5CboEl6jQ9sO9ful_M_orOfoA_bzpdBt0uiI0ExC8d8A_r8k1o</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1534149509</pqid></control><display><type>article</type><title>A Non-Forced-Write Atomic Commit Protocol for Cluster File Systems</title><source>ABI/INFORM Global</source><source>Springer Link</source><creator>邵冰清 张军伟 郑彩平 张浩 刘振军 许鲁</creator><creatorcontrib>邵冰清 张军伟 郑彩平 张浩 刘振军 许鲁</creatorcontrib><description>Distributed metadata consistency is one of the critical issues of metadata clusters in distributed file systems. Existing methods to maintain metadata consistency generally need several log forced write operations. Since synchronous disk IO is very inefficient, the average response time of metadata operations is greatly increased. In this paper, an asynchronous atomic commit protocol （ACP） named Dual-Log （DL） is presented. It does not need any log forced write operations. Optimizing for distributed metadata operations involving only two metadata servers, DL mutually records the redo log in counterpart metadata servers by transferring through the low latency network. A crashed metadata server can redo the metadata operation with the redundant redo log. Since the latency of the network is much lower than the latency of disk IO, DL can improve the performance of distributed metadata service significantly. The prototype of DL is implemented based on local journal. The performance is tested by comparing with two widely used protocols, EP and S2PC-MP, and the results show that the average response time of distributed metadata operations is reduced by about 40%-60%, and the recovery time is only I second under 10 thousands uncompleted distributed metadata operations.</description><identifier>ISSN: 1000-9000</identifier><identifier>EISSN: 1860-4749</identifier><identifier>DOI: 10.1007/s11390-014-1432-y</identifier><language>eng</language><publisher>Boston: Springer US</publisher><subject>Artificial Intelligence ; Clusters ; Computer Science ; Consistency ; Data Structures and Information Theory ; Disks ; Distributed processing ; Information Systems Applications (incl.Internet) ; Metadata ; Networks ; Protocol ; Protocol (computers) ; R&amp;D ; Regular Paper ; Research &amp; development ; Response time ; Science ; Servers ; Servers (computers) ; Software Engineering ; Studies ; Theory of Computation ; 元数据服务器 ; 分布式文件系统 ; 原子 ; 平均响应时间 ; 提交协议 ; 数据一致性 ; 数据操作 ; 群集</subject><ispartof>Journal of computer science and technology, 2014, Vol.29 (2), p.303-315</ispartof><rights>Springer Science+Business Media New York &amp; Science Press, China 2014</rights><rights>Copyright © Wanfang Data Co. Ltd. All Rights Reserved.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c364t-a14a965e7c3bda6335757472c17b075ba06f62f939ee96dde6f3fbe8f72199aa3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Uhttp://image.cqvip.com/vip1000/qk/85226X/85226X.jpg</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/1534149509?pq-origsite=primo$$EHTML$$P50$$Gproquest$$H</linktohtml><link.rule.ids>314,777,781,11669,27905,27906,36041,36042,44344</link.rule.ids></links><search><creatorcontrib>邵冰清 张军伟 郑彩平 张浩 刘振军 许鲁</creatorcontrib><title>A Non-Forced-Write Atomic Commit Protocol for Cluster File Systems</title><title>Journal of computer science and technology</title><addtitle>J. Comput. Sci. Technol</addtitle><addtitle>Journal of Computer Science and Technology</addtitle><description>Distributed metadata consistency is one of the critical issues of metadata clusters in distributed file systems. Existing methods to maintain metadata consistency generally need several log forced write operations. Since synchronous disk IO is very inefficient, the average response time of metadata operations is greatly increased. In this paper, an asynchronous atomic commit protocol （ACP） named Dual-Log （DL） is presented. It does not need any log forced write operations. Optimizing for distributed metadata operations involving only two metadata servers, DL mutually records the redo log in counterpart metadata servers by transferring through the low latency network. A crashed metadata server can redo the metadata operation with the redundant redo log. Since the latency of the network is much lower than the latency of disk IO, DL can improve the performance of distributed metadata service significantly. The prototype of DL is implemented based on local journal. The performance is tested by comparing with two widely used protocols, EP and S2PC-MP, and the results show that the average response time of distributed metadata operations is reduced by about 40%-60%, and the recovery time is only I second under 10 thousands uncompleted distributed metadata operations.</description><subject>Artificial Intelligence</subject><subject>Clusters</subject><subject>Computer Science</subject><subject>Consistency</subject><subject>Data Structures and Information Theory</subject><subject>Disks</subject><subject>Distributed processing</subject><subject>Information Systems Applications (incl.Internet)</subject><subject>Metadata</subject><subject>Networks</subject><subject>Protocol</subject><subject>Protocol (computers)</subject><subject>R&amp;D</subject><subject>Regular Paper</subject><subject>Research &amp; development</subject><subject>Response time</subject><subject>Science</subject><subject>Servers</subject><subject>Servers (computers)</subject><subject>Software Engineering</subject><subject>Studies</subject><subject>Theory of Computation</subject><subject>元数据服务器</subject><subject>分布式文件系统</subject><subject>原子</subject><subject>平均响应时间</subject><subject>提交协议</subject><subject>数据一致性</subject><subject>数据操作</subject><subject>群集</subject><issn>1000-9000</issn><issn>1860-4749</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2014</creationdate><recordtype>article</recordtype><sourceid>M0C</sourceid><recordid>eNp9kTFPwzAQhSMEEqXwA9iCWBgwnGPHrscSUUCqAAkQo-WkdklI4tZORfvvcRVUIQaW8w3fe-_kF0WnGK4wAL_2GBMBCDBFmJIEbfaiAR4xQJRTsR92AEAijMPoyPsKgHCgdBDdjONH26KJdYWeoXdXdjoed7YpizizTVN28bOznS1sHRvr4qxe-U67eFLWOn7ZhL3xx9GBUbXXJz_vMHqb3L5m92j6dPeQjaeoIIx2SGGqBEs1L0g-U4yQlKec8qTAPAee5gqYYYkRRGgt2GymmSEm1yPDEyyEUmQYXfa-X6o1qp3Lyq5cGxJl5avPdeXXudRJ-AEIgwT8oscXzi5X2neyKX2h61q12q68xGkCYkQSTgN6_gfdWeOUUExFCiJQuKcKZ7132siFKxvlNhKD3JYg-xJkOEFuS5CboEl6jQ9sO9ful_M_orOfoA_bzpdBt0uiI0ExC8d8A_r8k1o</recordid><startdate>2014</startdate><enddate>2014</enddate><creator>邵冰清 张军伟 郑彩平 张浩 刘振军 许鲁</creator><general>Springer US</general><general>Springer Nature B.V</general><general>University of Chinese Academy of Sciences, Beijing 100049, China</general><general>Data Storage and Management Technology Research Center, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China%Data Storage and Management Technology Research Center, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China</general><scope>2RA</scope><scope>92L</scope><scope>CQIGP</scope><scope>W92</scope><scope>~WA</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7SC</scope><scope>7WY</scope><scope>7WZ</scope><scope>7XB</scope><scope>87Z</scope><scope>8AL</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FK</scope><scope>8FL</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BEZIV</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FRNLG</scope><scope>F~G</scope><scope>GNUQQ</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K60</scope><scope>K6~</scope><scope>K7-</scope><scope>L.-</scope><scope>L6V</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>M0C</scope><scope>M0N</scope><scope>M7S</scope><scope>P5Z</scope><scope>P62</scope><scope>PQBIZ</scope><scope>PQBZA</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PTHSS</scope><scope>Q9U</scope><scope>2B.</scope><scope>4A8</scope><scope>92I</scope><scope>93N</scope><scope>PSX</scope><scope>TCJ</scope></search><sort><creationdate>2014</creationdate><title>A Non-Forced-Write Atomic Commit Protocol for Cluster File Systems</title><author>邵冰清 张军伟 郑彩平 张浩 刘振军 许鲁</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c364t-a14a965e7c3bda6335757472c17b075ba06f62f939ee96dde6f3fbe8f72199aa3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2014</creationdate><topic>Artificial Intelligence</topic><topic>Clusters</topic><topic>Computer Science</topic><topic>Consistency</topic><topic>Data Structures and Information Theory</topic><topic>Disks</topic><topic>Distributed processing</topic><topic>Information Systems Applications (incl.Internet)</topic><topic>Metadata</topic><topic>Networks</topic><topic>Protocol</topic><topic>Protocol (computers)</topic><topic>R&amp;D</topic><topic>Regular Paper</topic><topic>Research &amp; development</topic><topic>Response time</topic><topic>Science</topic><topic>Servers</topic><topic>Servers (computers)</topic><topic>Software Engineering</topic><topic>Studies</topic><topic>Theory of Computation</topic><topic>元数据服务器</topic><topic>分布式文件系统</topic><topic>原子</topic><topic>平均响应时间</topic><topic>提交协议</topic><topic>数据一致性</topic><topic>数据操作</topic><topic>群集</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>邵冰清 张军伟 郑彩平 张浩 刘振军 许鲁</creatorcontrib><collection>维普_期刊</collection><collection>中文科技期刊数据库-CALIS站点</collection><collection>中文科技期刊数据库-7.0平台</collection><collection>中文科技期刊数据库-工程技术</collection><collection>中文科技期刊数据库- 镜像站点</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Computer and Information Systems Abstracts</collection><collection>ABI/INFORM Collection</collection><collection>ABI/INFORM Global (PDF only)</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>ABI/INFORM Collection</collection><collection>Computing Database (Alumni Edition)</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ABI/INFORM Collection (Alumni Edition)</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>Advanced Technologies &amp; Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Business Premium Collection</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>Business Premium Collection (Alumni)</collection><collection>ABI/INFORM Global (Corporate)</collection><collection>ProQuest Central Student</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>ProQuest Business Collection (Alumni Edition)</collection><collection>ProQuest Business Collection</collection><collection>Computer Science Database</collection><collection>ABI/INFORM Professional Advanced</collection><collection>ProQuest Engineering Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>ABI/INFORM Global</collection><collection>Computing Database</collection><collection>Engineering Database</collection><collection>Advanced Technologies &amp; Aerospace Database</collection><collection>ProQuest Advanced Technologies &amp; Aerospace Collection</collection><collection>ProQuest One Business</collection><collection>ProQuest One Business (Alumni)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>Engineering Collection</collection><collection>ProQuest Central Basic</collection><collection>Wanfang Data Journals - Hong Kong</collection><collection>WANFANG Data Centre</collection><collection>Wanfang Data Journals</collection><collection>万方数据期刊 - 香港版</collection><collection>China Online Journals (COJ)</collection><collection>China Online Journals (COJ)</collection><jtitle>Journal of computer science and technology</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>邵冰清 张军伟 郑彩平 张浩 刘振军 许鲁</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A Non-Forced-Write Atomic Commit Protocol for Cluster File Systems</atitle><jtitle>Journal of computer science and technology</jtitle><stitle>J. Comput. Sci. Technol</stitle><addtitle>Journal of Computer Science and Technology</addtitle><date>2014</date><risdate>2014</risdate><volume>29</volume><issue>2</issue><spage>303</spage><epage>315</epage><pages>303-315</pages><issn>1000-9000</issn><eissn>1860-4749</eissn><abstract>Distributed metadata consistency is one of the critical issues of metadata clusters in distributed file systems. Existing methods to maintain metadata consistency generally need several log forced write operations. Since synchronous disk IO is very inefficient, the average response time of metadata operations is greatly increased. In this paper, an asynchronous atomic commit protocol （ACP） named Dual-Log （DL） is presented. It does not need any log forced write operations. Optimizing for distributed metadata operations involving only two metadata servers, DL mutually records the redo log in counterpart metadata servers by transferring through the low latency network. A crashed metadata server can redo the metadata operation with the redundant redo log. Since the latency of the network is much lower than the latency of disk IO, DL can improve the performance of distributed metadata service significantly. The prototype of DL is implemented based on local journal. The performance is tested by comparing with two widely used protocols, EP and S2PC-MP, and the results show that the average response time of distributed metadata operations is reduced by about 40%-60%, and the recovery time is only I second under 10 thousands uncompleted distributed metadata operations.</abstract><cop>Boston</cop><pub>Springer US</pub><doi>10.1007/s11390-014-1432-y</doi><tpages>13</tpages></addata></record>
fulltext fulltext
identifier ISSN: 1000-9000
ispartof Journal of computer science and technology, 2014, Vol.29 (2), p.303-315
issn 1000-9000
1860-4749
language eng
recordid cdi_wanfang_journals_jsjkxjsxb_e201402013
source ABI/INFORM Global; Springer Link
subjects Artificial Intelligence
Clusters
Computer Science
Consistency
Data Structures and Information Theory
Disks
Distributed processing
Information Systems Applications (incl.Internet)
Metadata
Networks
Protocol
Protocol (computers)
R&D
Regular Paper
Research & development
Response time
Science
Servers
Servers (computers)
Software Engineering
Studies
Theory of Computation
元数据服务器
分布式文件系统
原子
平均响应时间
提交协议
数据一致性
数据操作
群集
title A Non-Forced-Write Atomic Commit Protocol for Cluster File Systems
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-19T18%3A01%3A27IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-wanfang_jour_proqu&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20Non-Forced-Write%20Atomic%20Commit%20Protocol%20for%20Cluster%20File%20Systems&rft.jtitle=Journal%20of%20computer%20science%20and%20technology&rft.au=%E9%82%B5%E5%86%B0%E6%B8%85%20%E5%BC%A0%E5%86%9B%E4%BC%9F%20%E9%83%91%E5%BD%A9%E5%B9%B3%20%E5%BC%A0%E6%B5%A9%20%E5%88%98%E6%8C%AF%E5%86%9B%20%E8%AE%B8%E9%B2%81&rft.date=2014&rft.volume=29&rft.issue=2&rft.spage=303&rft.epage=315&rft.pages=303-315&rft.issn=1000-9000&rft.eissn=1860-4749&rft_id=info:doi/10.1007/s11390-014-1432-y&rft_dat=%3Cwanfang_jour_proqu%3Ejsjkxjsxb_e201402013%3C/wanfang_jour_proqu%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c364t-a14a965e7c3bda6335757472c17b075ba06f62f939ee96dde6f3fbe8f72199aa3%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=1534149509&rft_id=info:pmid/&rft_cqvip_id=48941609&rft_wanfj_id=jsjkxjsxb_e201402013&rfr_iscdi=true