VQA: Visual Question Answering: www.visualqa.org

We propose the task of free-form and open-ended Visual Question Answering (VQA). Given an image and a natural language question about the image, the task is to provide an accurate natural language answer. Mirroring real-world scenarios, such as helping the visually impaired, both the questions and a...

Full description

Saved in:
Bibliographic Details
Published in:International journal of computer vision 2017-05, Vol.123 (1), p.4-31
Main Authors: Agrawal, Aishwarya, Lu, Jiasen, Antol, Stanislaw, Mitchell, Margaret, Zitnick, C. Lawrence, Parikh, Devi, Batra, Dhruv
Format: Article
Language:English
Subjects:
Algorithms
Artificial Intelligence
Computer Imaging
Computer Science
Computer vision
Datasets
Human performance
Image Processing and Computer Vision
Image processing systems
Information dissemination
Language
Natural language
Natural language (computers)
Natural language processing
Pattern Recognition
Pattern Recognition and Graphics
Pizza
Questioning
Studies
Tasks
Texts
Vision
Vision systems
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose the task of free-form and open-ended Visual Question Answering (VQA). Given an image and a natural language question about the image, the task is to provide an accurate natural language answer. Mirroring real-world scenarios, such as helping the visually impaired, both the questions and answers are open-ended. Visual questions selectively target different areas of an image, including background details and underlying context. As a result, a system that succeeds at VQA typically needs a more detailed understanding of the image and complex reasoning than a system producing generic image captions. Moreover, VQA is amenable to automatic evaluation, since many open-ended answers contain only a few words or a closed set of answers that can be provided in a multiple-choice format. We provide a dataset containing ∼ 0.25 M images, ∼ 0.76 M questions, and ∼ 10 M answers ( www.visualqa.org ), and discuss the information it provides. Numerous baselines and methods for VQA are provided and compared with human performance. Our VQA demo is available on CloudCV ( http://cloudcv.org/vqa ).
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-016-0966-6