TADOC: Text analytics directly on compression

This article provides a comprehensive description of text analytics directly on compression (TADOC), which enables direct document analytics on compressed textual data. The article explains the concept of TADOC and the challenges to its effective realizations. Additionally, a series of guidelines an...

Full description

Saved in:
Bibliographic Details
Published in:The VLDB journal 2021-03, Vol.30 (2), p.163-188
Main Authors: Zhang, Feng, Zhai, Jidong, Shen, Xipeng, Wang, Dalin, Chen, Zheng, Mutlu, Onur, Chen, Wenguang, Du, Xiaoyong
Format: Article
Language:English
Subjects:
Algorithms
Computer Science
Data processing
Data structures
Database Management
Mathematical analysis
Regular Paper
Task complexity
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This article provides a comprehensive description of text analytics directly on compression (TADOC), which enables direct document analytics on compressed textual data. The article explains the concept of TADOC and the challenges to its effective realizations. Additionally, a series of guidelines and technical solutions that effectively address those challenges, including the adoption of a hierarchical compression method and a set of novel algorithms and data structure designs, are presented. Experiments on six data analytics tasks of various complexities show that TADOC can save 90.8% storage space and 87.9% memory usage, while halving data processing times.
ISSN:1066-8888
0949-877X
DOI:10.1007/s00778-020-00636-3