Embracing data abundance: BookTest Dataset for Reading Comprehension

There is a practically unlimited amount of natural language data available. Still, recent work in text comprehension has focused on datasets which are small relative to current computing possibilities. This article is making a case for the community to move to larger data and as a step in that direc...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2016-10
Main Authors: Bajgar, Ondrej, Kadlec, Rudolf, Kleindienst, Jan
Format: Article
Language:English
Subjects:
Datasets
Model accuracy
Reading comprehension
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:There is a practically unlimited amount of natural language data available. Still, recent work in text comprehension has focused on datasets which are small relative to current computing possibilities. This article is making a case for the community to move to larger data and as a step in that direction it is proposing the BookTest, a new dataset similar to the popular Children's Book Test (CBT), however more than 60 times larger. We show that training on the new data improves the accuracy of our Attention-Sum Reader model on the original CBT test data by a much larger margin than many recent attempts to improve the model architecture. On one version of the dataset our ensemble even exceeds the human baseline provided by Facebook. We then show in our own human study that there is still space for further improvement.
ISSN:2331-8422