Predicate Correlation Learning for Scene Graph Generation

For a typical Scene Graph Generation (SGG) method in image understanding, there usually exists a large gap in the performance of the predicates' head classes and tail classes. This phenomenon is mainly caused by the semantic overlap between different predicates as well as the long-tailed data d...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on image processing 2022-01, Vol.31, p.4173-4185
Main Authors: Tao, Leitian, Mi, Li, Li, Nannan, Cheng, Xianhang, Hu, Yaosi, Chen, Zhenzhong
Format: Article
Language:English
Subjects:
Correlation
Correlation analysis
Head
Image understanding
Learning
long-tailed bias
Optimization
Phase change materials
predicate correlation
scene graph generation
semantic overlap
Semantics
Tail
Visualization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:For a typical Scene Graph Generation (SGG) method in image understanding, there usually exists a large gap in the performance of the predicates' head classes and tail classes. This phenomenon is mainly caused by the semantic overlap between different predicates as well as the long-tailed data distribution. In this paper, a Predicate Correlation Learning (PCL) method for SGG is proposed to address the above problems by taking the correlation between predicates into consideration. To measure the semantic overlap between highly correlated predicate classes, a Predicate Correlation Matrix (PCM) is defined to quantify the relationship between predicate pairs, which is dynamically updated to remove the matrix's long-tailed bias. In addition, PCM is integrated into a predicate correlation loss function ( L_{PC} ) to reduce discouraging gradients of unannotated classes. The proposed method is evaluated on several benchmarks, where the performance of the tail classes is significantly improved when built on existing methods.
ISSN:1057-7149
1941-0042
DOI:10.1109/TIP.2022.3181511