AnaMeta: A Table Understanding Dataset of Field Metadata Knowledge Shared by Multi-dimensional Data Analysis Tasks

Tabular data analysis is performed every day across various domains. It requires an accurate understanding of field semantics to correctly operate on table fields and find common patterns in daily analysis. In this paper, we introduce the AnaMeta dataset, a collection of 467k tables with derived sup...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-05
Main Authors: He, Xinyi, Zhou, Mengyu, Zhou, Mingjie, Xu, Jialiang, Lv, Xiao, Li, Tianle, Shao, Yijia, Shi, Han, Yuan, Zejian, Zhang, Dongmei
Format: Article
Language:English
Subjects:
Accuracy
Data analysis
Inference
Knowledge representation
Machine learning
Metadata
Model accuracy
Multidimensional data
Multitasking
Tables (data)
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Tabular data analysis is performed every day across various domains. It requires an accurate understanding of field semantics to correctly operate on table fields and find common patterns in daily analysis. In this paper, we introduce the AnaMeta dataset, a collection of 467k tables with derived supervision labels for four types of commonly used field metadata: measure/dimension dichotomy, common field roles, semantic field type, and default aggregation function. We evaluate a wide range of models for inferring metadata as the benchmark. We also propose a multi-encoder framework, called KDF, which improves the metadata understanding capability of tabular models by incorporating distribution and knowledge information. Furthermore, we propose four interfaces for incorporating field metadata into downstream analysis tasks.
ISSN:2331-8422