A New Training Data Organization Form and Training Mode for Unbiased Scene Graph Generation

The current mainstream studies on Scene Graph Generation (SGG) devote to the long-tailed predicate distribution problem to generate unbiased scene graph. The long-tailed predicate distribution exists in VG dataset and is more severe during the SGG network training process. Most existing de-biasing m...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on circuits and systems for video technology 2024-07, Vol.34 (7), p.5295-5305
Main Authors: Xu, Hongbo, Wang, Lichun, Xu, Kai, Fu, Fangyu, Yin, Baocai, Huang, Qingming
Format: Article
Language:English
Subjects:
balanced predicate instances
Head
new training data organization form
new training mode
Predictive models
Proposals
Sampling
Semantics
Tail
Training
Training data
Unbiased scene graph generation
Weighting
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The current mainstream studies on Scene Graph Generation (SGG) devote to the long-tailed predicate distribution problem to generate unbiased scene graph. The long-tailed predicate distribution exists in VG dataset and is more severe during the SGG network training process. Most existing de-biasing methods solve the problem by applying re- sampling or re- weighting in a mini-batch, with the main idea being to provide unbiased attention to different predicate categories based on prior predicate distributions. During the training process of SGG models, existing training mode samples several images into a mini-batch to obtain training data, thus providing sparse and scattered predicate instances for training. However, sampling predicate instances from a limited set of predicate samples in terms of quantity and category poses difficulties in training unbiased SGG models. In order to provide a wider range for sampling predicate instances, this paper reorganizes the images in VG training set with a new form, i.e. object-pairs, and constructs VG-OP (VG Object-Pair) training set to save object-pairs. Meanwhile, this paper introduces a new SGG network training mode, which can realize unbiased SGG without re- sampling or re- weighting. In particular, a Predicate-balanced Sampling Network (PS-Net) is designed to validate the new training mode. Extensive experiments on VG test set demonstrate that our method achieves competitive or state-of-the-art unbiased SGG performance.
ISSN:1051-8215
1558-2205
DOI:10.1109/TCSVT.2023.3344569