Mimir: Bringing CTables into Practice

The present state of the art in analytics requires high upfront investment of human effort and computational resources to curate datasets, even before the first query is posed. So-called pay-as-you-go data curation techniques allow these high costs to be spread out, first by enabling queries over un...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2016-01
Main Authors: Nandi, Arindam, Yang, Ying, Kennedy, Oliver, Glavic, Boris, Fehling, Ronny, Liu, Zhen Hua, Gawlick, Dieter
Format: Article
Language:English
Subjects:
Graphical user interface
Interactive systems
Quality assessment
Queries
Query processing
Uncertainty
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The present state of the art in analytics requires high upfront investment of human effort and computational resources to curate datasets, even before the first query is posed. So-called pay-as-you-go data curation techniques allow these high costs to be spread out, first by enabling queries over uncertain and incomplete data, and then by assessing the quality of the query results. We describe the design of a system, called Mimir, around a recently introduced class of probabilistic pay-as-you-go data cleaning operators called Lenses. Mimir wraps around any deterministic database engine using JDBC, extending it with support for probabilistic query processing. Queries processed through Mimir produce uncertainty-annotated result cursors that allow client applications to quickly assess result quality and provenance. We also present a GUI that provides analysts with an interactive tool for exploring the uncertainty exposed by the system. Finally, we present optimizations that make Lenses scalable, and validate this claim through experimental evidence.
ISSN:2331-8422