FeTaQA: Free-form Table Question Answering

Existing table question answering datasets contain abundant factual questions that primarily evaluate the query and schema comprehension capability of a system, but they fail to include questions that require complex reasoning and integration of information due to the constraint of the associated sh...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-04
Main Authors: Linyong Nan, Hsieh, Chiachun, Mao, Ziming, Lin, Xi Victoria, Verma, Neha, Zhang, Rui, Kryściński, Wojciech, Schoelkopf, Nick, Riley, Kong, Tang, Xiangru, Mutuma, Murori, Rosand, Ben, Trindade, Isabel, Bandaru, Renusree, Cunningham, Jacob, Xiong, Caiming, Radev, Dragomir
Format: Article
Language:English
Subjects:
Datasets
Free form
Questions
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Existing table question answering datasets contain abundant factual questions that primarily evaluate the query and schema comprehension capability of a system, but they fail to include questions that require complex reasoning and integration of information due to the constraint of the associated short-form answers. To address these issues and to demonstrate the full challenge of table question answering, we introduce FeTaQA, a new dataset with 10K Wikipedia-based {table, question, free-form answer, supporting table cells} pairs. FeTaQA yields a more challenging table question answering setting because it requires generating free-form text answers after retrieval, inference, and integration of multiple discontinuous facts from a structured knowledge source. Unlike datasets of generative QA over text in which answers are prevalent with copies of short text spans from the source, answers in our dataset are human-generated explanations involving entities and their high-level relations. We provide two benchmark methods for the proposed task: a pipeline method based on semantic-parsing-based QA systems and an end-to-end method based on large pretrained text generation models, and show that FeTaQA poses a challenge for both methods.
ISSN:2331-8422