Data Lake, Data Warehouse, Datamart, and Feature Store: Their Contributions to the Complete Data Reuse Pipeline

The growing adoption and use of health information technology has generated a wealth of clinical data in electronic format, offering opportunities for data reuse beyond direct patient care. However, as data are distributed across multiple software, it becomes challenging to cross-reference informati...

Full description

Saved in:
Bibliographic Details
Published in:JMIR medical informatics 2024-07, Vol.12, p.e54590-e54590
Main Authors: Lamer, Antoine, Saint-Dizier, Chloé, Paris, Nicolas, Chazard, Emmanuel
Format: Article
Language:English
Subjects:
Clinical Informatics
Collaboration
Data analysis
Data collection
Data models
Data Science
Data warehouses
eHealth Infrastructures
Electronic health records
Health Information Exchange
Information Models
Information systems
Life Sciences
Metadata
Personal health
Python
Secondary Use of Clinical Data for Research and Surveillance
Software upgrading
Standards and Interoperability
Viewpoint
Viewpoints on and Experiences with Digital Technologies in Health
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The growing adoption and use of health information technology has generated a wealth of clinical data in electronic format, offering opportunities for data reuse beyond direct patient care. However, as data are distributed across multiple software, it becomes challenging to cross-reference information between sources due to differences in formats, vocabularies, and technologies and the absence of common identifiers among software. To address these challenges, hospitals have adopted data warehouses to consolidate and standardize these data for research. Additionally, as a complement or alternative, data lakes store both source data and metadata in a detailed and unprocessed format, empowering exploration, manipulation, and adaptation of the data to meet specific analytical needs. Subsequently, datamarts are used to further refine data into usable information tailored to specific research questions. However, for efficient analysis, a feature store is essential to pivot and denormalize the data, simplifying queries. In conclusion, while data warehouses are crucial, data lakes, datamarts, and feature stores play essential and complementary roles in facilitating data reuse for research and analysis in health care.
ISSN:2291-9694
2291-9694
DOI:10.2196/54590