Causal Interventional Training for Image Recognition

Deep learning models often fit undesired dataset bias in training. In this paper, we formulate the bias using causal inference , which helps us uncover the ever-elusive causalities among the key factors in training, and thus pursue the desired causal effect without the bias. We start from revisiting...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on multimedia 2023, Vol.25, p.1033-1044
Main Authors: Qin, Wei, Zhang, Hanwang, Hong, Richang, Lim, Ee-Peng, Sun, Qianru
Format: Article
Language:English
Subjects:
Artificial neural networks
Automobiles
Bias
causal intervention
Causality
Context
Data models
Datasets
deep learning
Deserts
Image recognition
imagenet
Machine learning
Object recognition
Road transportation
Task analysis
Training
Visualization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Deep learning models often fit undesired dataset bias in training. In this paper, we formulate the bias using causal inference , which helps us uncover the ever-elusive causalities among the key factors in training, and thus pursue the desired causal effect without the bias. We start from revisiting the process of building a visual recognition system, and then propose a structural causal model (SCM) for the key variables involved in dataset collection and recognition model: object, common sense, bias, context, and label prediction. Based on the SCM, one can observe that there are "good" and "bad" biases. Intuitively, in the image where a car is driving on a high way in a desert, the "good" bias denoting the common-sense context is the highway, and the "bad" bias accounting for the noisy context factor is the desert. We tackle this problem with a novel causal interventional training ( CIT ) approach, where we control the observed context in each object class. We offer theoretical justifications for CIT and validate it with extensive classification experiments on CIFAR-10, CIFAR-100 and ImageNet, e.g. , surpassing the standard deep neural networks ResNet-34 and ResNet-50, respectively, by 0.95% and 0.70% accuracies on the ImageNet. Our code is open-sourced on the GitHub https://github.com/qinwei-hfut/CIT .
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2021.3136717