Fair-VQA: Fairness-Aware Visual Question Answering Through Sensitive Attribute Prediction

Visual Question Answering (VQA) is a task that answers questions on given images. Although previous works achieve a great improvement in VQA performance, they do not consider the fairness of answers in terms of ethically sensitive attributes, such as gender. Therefore, we propose a Fair-VQA model th...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2020, Vol.8, p.215091-215099
Main Authors: Park, Sungho, Hwang, Sunhee, Hong, Jongkwang, Byun, Hyeran
Format: Article
Language:English
Subjects:
Artificial intelligence
Data models
Face recognition
FAI
fairness
Knowledge discovery
Modules
Performance degradation
Predictive models
Questions
Task analysis
visual question answering
Visualization
VQA
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Visual Question Answering (VQA) is a task that answers questions on given images. Although previous works achieve a great improvement in VQA performance, they do not consider the fairness of answers in terms of ethically sensitive attributes, such as gender. Therefore, we propose a Fair-VQA model that contains two modules: VQA module and SAP (Sensitive Attribute Prediction) module. On top of VQA module, which predicts various kinds of answers, SAP module predicts only sensitive attributes using the same inputs. The predictions of SAP module are utilized to rectify answers from VQA module to be fairer in terms of the sensitive attributes with graceful performance degradation. To validate the proposed method, we conduct extensive experiments on VQA, GQA, and our proposing VQA-Gender datasets. In all the experiments, our method shows the fairest results in various metrics for fairness. Moreover, we demonstrate that our method works interpretably through the analysis of visualized attention maps.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2020.3041503