Towards Migration-Free "Just-in-Case" Data Archival for Future Cloud Data Lakes Using Synthetic DNA

Given the growing adoption of AI, cloud data lakes are facing the need to support cost-effective "just-in-case" data archival over long time periods to meet regulatory compliance requirements. Unfortunately, current media technologies suffer from fundamental issues that will soon, if not a...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings of the VLDB Endowment 2023-04, Vol.16 (8), p.1923-1929
Main Authors: Marinelli, Eugenio, Yan, Yiqing, Magnone, Virginie, Dumargne, Charlotte, Barbry, Pascal, Heinis, Thomas, Appuswamy, Raja
Format: Article
Language:English
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Given the growing adoption of AI, cloud data lakes are facing the need to support cost-effective "just-in-case" data archival over long time periods to meet regulatory compliance requirements. Unfortunately, current media technologies suffer from fundamental issues that will soon, if not already, make cost-effective data archival infeasible. In this paper, we present a vision for redesigning the archival tier of cloud data lakes based on a novel, obsolescence-free storage medium-synthetic DNA. In doing so, we make two contributions: (i) we highlight the challenges in using DNA for data archival and list several open research problems, (ii) we outline OligoArchive-DSM (OA-DSM)-an end-to-end DNA storage pipeline that we are developing to demonstrate the feasibility of our vision.
ISSN:2150-8097
2150-8097
DOI:10.14778/3594512.3594522