Lite2: A Schemaless Zero-Copy Serialization Format

In the field of data transmission and storage, serialization formats play a crucial role by converting complex data structures into a byte stream that can be easily stored, transmitted, and reconstructed. Despite the myriad available serialization formats, ranging from JSON to Protobuf, each has lim...

Full description

Saved in:
Bibliographic Details
Published in:Computers (Basel) 2024-04, Vol.13 (4), p.89
Main Authors: Chen, Tianyi, Guan, Xiaotong, Shuai, Shi, Huang, Cuiting, Aibin, Michal
Format: Article
Language:English
Subjects:
B trees
Communication
Copying
data formats
data serialization
Data structures
Data transmission
Flexibility
Format
JavaScript
Popularity
Programming languages
Protocol
Python
schemaless
zero-copy
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites cdi_FETCH-LOGICAL-c264t-acb0697cd6f6c9dda613bce9b6462e011ac34c10b8a1d00576710ea3b477343f3
container_end_page
container_issue 4
container_start_page 89
container_title Computers (Basel)
container_volume 13
creator Chen, Tianyi
Guan, Xiaotong
Shuai, Shi
Huang, Cuiting
Aibin, Michal
description In the field of data transmission and storage, serialization formats play a crucial role by converting complex data structures into a byte stream that can be easily stored, transmitted, and reconstructed. Despite the myriad available serialization formats, ranging from JSON to Protobuf, each has limitations, particularly in balancing schema flexibility, performance, and data copying overhead. This paper introduces Lite2, a novel data serialization format that addresses these challenges by combining schemaless flexibility with the efficiency of zero-copy operations for flat or key–value pair data types. Unlike traditional formats that often require a predefined schema and involve significant data copying during serialization and deserialization, Lite2 offers a dynamic schemaless approach that eliminates unnecessary data copying, optimizing system performance and efficiency. Built upon a contiguously stored B-tree structure, Lite2 enables efficient data lookup and modification without deserialization, thereby achieving zero-copy operations.
doi_str_mv 10.3390/computers13040089
format article
fullrecord <record><control><sourceid>proquest_doaj_</sourceid><recordid>TN_cdi_doaj_primary_oai_doaj_org_article_e1bd9a4fd762405a8bef96e00585f1e6</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><doaj_id>oai_doaj_org_article_e1bd9a4fd762405a8bef96e00585f1e6</doaj_id><sourcerecordid>3046717751</sourcerecordid><originalsourceid>FETCH-LOGICAL-c264t-acb0697cd6f6c9dda613bce9b6462e011ac34c10b8a1d00576710ea3b477343f3</originalsourceid><addsrcrecordid>eNplkEFLw0AQhRdRsNT-AG8Bz9Gd7GY3660Uq0LBQxXEyzLZTDQl6cbd9FB_vdGKCM5lhuHxvcdj7Bz4pRCGXznf9buBQgTBJeeFOWKTjGuRSgHPx3_uUzaLccPHMSCKDCYsWzUDZdfJPFm7N-qwpRiTFwo-Xfh-n6wpNNg2Hzg0fpssfehwOGMnNbaRZj97yp6WN4-Lu3T1cHu_mK9Slyk5pOhKrox2laqVM1WFCkTpyJRKqow4ADohHfCyQKg4z7XSwAlFKbUWUtRiyu4P3Mrjxvah6TDsrcfGfj98eLUYhsa1ZAnKyqCsK60yyXMsSqqNopFa5DWQGlkXB1Yf_PuO4mA3fhe2Y3w7VjY6a53DqIKDygUfY6D61xW4_Wra_mtafAKjVXG4</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3046717751</pqid></control><display><type>article</type><title>Lite2: A Schemaless Zero-Copy Serialization Format</title><source>Publicly Available Content Database</source><creator>Chen, Tianyi ; Guan, Xiaotong ; Shuai, Shi ; Huang, Cuiting ; Aibin, Michal</creator><creatorcontrib>Chen, Tianyi ; Guan, Xiaotong ; Shuai, Shi ; Huang, Cuiting ; Aibin, Michal</creatorcontrib><description>In the field of data transmission and storage, serialization formats play a crucial role by converting complex data structures into a byte stream that can be easily stored, transmitted, and reconstructed. Despite the myriad available serialization formats, ranging from JSON to Protobuf, each has limitations, particularly in balancing schema flexibility, performance, and data copying overhead. This paper introduces Lite2, a novel data serialization format that addresses these challenges by combining schemaless flexibility with the efficiency of zero-copy operations for flat or key–value pair data types. Unlike traditional formats that often require a predefined schema and involve significant data copying during serialization and deserialization, Lite2 offers a dynamic schemaless approach that eliminates unnecessary data copying, optimizing system performance and efficiency. Built upon a contiguously stored B-tree structure, Lite2 enables efficient data lookup and modification without deserialization, thereby achieving zero-copy operations.</description><identifier>ISSN: 2073-431X</identifier><identifier>EISSN: 2073-431X</identifier><identifier>DOI: 10.3390/computers13040089</identifier><language>eng</language><publisher>Basel: MDPI AG</publisher><subject>B trees ; Communication ; Copying ; data formats ; data serialization ; Data structures ; Data transmission ; Flexibility ; Format ; JavaScript ; Popularity ; Programming languages ; Protocol ; Python ; schemaless ; zero-copy</subject><ispartof>Computers (Basel), 2024-04, Vol.13 (4), p.89</ispartof><rights>2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c264t-acb0697cd6f6c9dda613bce9b6462e011ac34c10b8a1d00576710ea3b477343f3</cites><orcidid>0000-0001-9800-834X</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.proquest.com/docview/3046717751/fulltextPDF?pq-origsite=primo$$EPDF$$P50$$Gproquest$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://www.proquest.com/docview/3046717751?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>314,780,784,25753,27924,27925,37012,44590,75126</link.rule.ids></links><search><creatorcontrib>Chen, Tianyi</creatorcontrib><creatorcontrib>Guan, Xiaotong</creatorcontrib><creatorcontrib>Shuai, Shi</creatorcontrib><creatorcontrib>Huang, Cuiting</creatorcontrib><creatorcontrib>Aibin, Michal</creatorcontrib><title>Lite2: A Schemaless Zero-Copy Serialization Format</title><title>Computers (Basel)</title><description>In the field of data transmission and storage, serialization formats play a crucial role by converting complex data structures into a byte stream that can be easily stored, transmitted, and reconstructed. Despite the myriad available serialization formats, ranging from JSON to Protobuf, each has limitations, particularly in balancing schema flexibility, performance, and data copying overhead. This paper introduces Lite2, a novel data serialization format that addresses these challenges by combining schemaless flexibility with the efficiency of zero-copy operations for flat or key–value pair data types. Unlike traditional formats that often require a predefined schema and involve significant data copying during serialization and deserialization, Lite2 offers a dynamic schemaless approach that eliminates unnecessary data copying, optimizing system performance and efficiency. Built upon a contiguously stored B-tree structure, Lite2 enables efficient data lookup and modification without deserialization, thereby achieving zero-copy operations.</description><subject>B trees</subject><subject>Communication</subject><subject>Copying</subject><subject>data formats</subject><subject>data serialization</subject><subject>Data structures</subject><subject>Data transmission</subject><subject>Flexibility</subject><subject>Format</subject><subject>JavaScript</subject><subject>Popularity</subject><subject>Programming languages</subject><subject>Protocol</subject><subject>Python</subject><subject>schemaless</subject><subject>zero-copy</subject><issn>2073-431X</issn><issn>2073-431X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><sourceid>DOA</sourceid><recordid>eNplkEFLw0AQhRdRsNT-AG8Bz9Gd7GY3660Uq0LBQxXEyzLZTDQl6cbd9FB_vdGKCM5lhuHxvcdj7Bz4pRCGXznf9buBQgTBJeeFOWKTjGuRSgHPx3_uUzaLccPHMSCKDCYsWzUDZdfJPFm7N-qwpRiTFwo-Xfh-n6wpNNg2Hzg0fpssfehwOGMnNbaRZj97yp6WN4-Lu3T1cHu_mK9Slyk5pOhKrox2laqVM1WFCkTpyJRKqow4ADohHfCyQKg4z7XSwAlFKbUWUtRiyu4P3Mrjxvah6TDsrcfGfj98eLUYhsa1ZAnKyqCsK60yyXMsSqqNopFa5DWQGlkXB1Yf_PuO4mA3fhe2Y3w7VjY6a53DqIKDygUfY6D61xW4_Wra_mtafAKjVXG4</recordid><startdate>20240401</startdate><enddate>20240401</enddate><creator>Chen, Tianyi</creator><creator>Guan, Xiaotong</creator><creator>Shuai, Shi</creator><creator>Huang, Cuiting</creator><creator>Aibin, Michal</creator><general>MDPI AG</general><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7SC</scope><scope>7XB</scope><scope>8AL</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FK</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>GNUQQ</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K7-</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>M0N</scope><scope>P5Z</scope><scope>P62</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0001-9800-834X</orcidid></search><sort><creationdate>20240401</creationdate><title>Lite2: A Schemaless Zero-Copy Serialization Format</title><author>Chen, Tianyi ; Guan, Xiaotong ; Shuai, Shi ; Huang, Cuiting ; Aibin, Michal</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c264t-acb0697cd6f6c9dda613bce9b6462e011ac34c10b8a1d00576710ea3b477343f3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>B trees</topic><topic>Communication</topic><topic>Copying</topic><topic>data formats</topic><topic>data serialization</topic><topic>Data structures</topic><topic>Data transmission</topic><topic>Flexibility</topic><topic>Format</topic><topic>JavaScript</topic><topic>Popularity</topic><topic>Programming languages</topic><topic>Protocol</topic><topic>Python</topic><topic>schemaless</topic><topic>zero-copy</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Chen, Tianyi</creatorcontrib><creatorcontrib>Guan, Xiaotong</creatorcontrib><creatorcontrib>Shuai, Shi</creatorcontrib><creatorcontrib>Huang, Cuiting</creatorcontrib><creatorcontrib>Aibin, Michal</creatorcontrib><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Computer and Information Systems Abstracts</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Computing Database (Alumni Edition)</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>Advanced Technologies &amp; Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>AUTh Library subscriptions: ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>ProQuest Central Student</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>Computer Science Database</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>Computing Database</collection><collection>Advanced Technologies &amp; Aerospace Database</collection><collection>ProQuest Advanced Technologies &amp; Aerospace Collection</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>Computers (Basel)</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Chen, Tianyi</au><au>Guan, Xiaotong</au><au>Shuai, Shi</au><au>Huang, Cuiting</au><au>Aibin, Michal</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Lite2: A Schemaless Zero-Copy Serialization Format</atitle><jtitle>Computers (Basel)</jtitle><date>2024-04-01</date><risdate>2024</risdate><volume>13</volume><issue>4</issue><spage>89</spage><pages>89-</pages><issn>2073-431X</issn><eissn>2073-431X</eissn><abstract>In the field of data transmission and storage, serialization formats play a crucial role by converting complex data structures into a byte stream that can be easily stored, transmitted, and reconstructed. Despite the myriad available serialization formats, ranging from JSON to Protobuf, each has limitations, particularly in balancing schema flexibility, performance, and data copying overhead. This paper introduces Lite2, a novel data serialization format that addresses these challenges by combining schemaless flexibility with the efficiency of zero-copy operations for flat or key–value pair data types. Unlike traditional formats that often require a predefined schema and involve significant data copying during serialization and deserialization, Lite2 offers a dynamic schemaless approach that eliminates unnecessary data copying, optimizing system performance and efficiency. Built upon a contiguously stored B-tree structure, Lite2 enables efficient data lookup and modification without deserialization, thereby achieving zero-copy operations.</abstract><cop>Basel</cop><pub>MDPI AG</pub><doi>10.3390/computers13040089</doi><orcidid>https://orcid.org/0000-0001-9800-834X</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 2073-431X
ispartof Computers (Basel), 2024-04, Vol.13 (4), p.89
issn 2073-431X
2073-431X
language eng
recordid cdi_doaj_primary_oai_doaj_org_article_e1bd9a4fd762405a8bef96e00585f1e6
source Publicly Available Content Database
subjects B trees
Communication
Copying
data formats
data serialization
Data structures
Data transmission
Flexibility
Format
JavaScript
Popularity
Programming languages
Protocol
Python
schemaless
zero-copy
title Lite2: A Schemaless Zero-Copy Serialization Format
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-27T01%3A46%3A05IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_doaj_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Lite2:%20A%20Schemaless%20Zero-Copy%20Serialization%20Format&rft.jtitle=Computers%20(Basel)&rft.au=Chen,%20Tianyi&rft.date=2024-04-01&rft.volume=13&rft.issue=4&rft.spage=89&rft.pages=89-&rft.issn=2073-431X&rft.eissn=2073-431X&rft_id=info:doi/10.3390/computers13040089&rft_dat=%3Cproquest_doaj_%3E3046717751%3C/proquest_doaj_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c264t-acb0697cd6f6c9dda613bce9b6462e011ac34c10b8a1d00576710ea3b477343f3%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=3046717751&rft_id=info:pmid/&rfr_iscdi=true