SRSG and S2SG: A Model and a Dataset for Scene Graph Generation of Remote Sensing Images from Segmentation Results

Remote sensing image analysis has drawn more attentions in the field of computer vision. At present, the methods commonly used in remote sensing image analysis are mainly at a low level semantically. The scene graph is an abstraction of objects and their relationships, which is a high level image un...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on geoscience and remote sensing 2022, Vol.60, p.1-1
Main Authors: Lin, Zhiyuan, Zhu, Feng, Kong, Yanzi, Wang, Qun, Wang, Jianyu
Format: Article
Language:English
Subjects:
Analysis
Computer vision
dataset
Datasets
Graphs
Image analysis
Image processing
Image segmentation
Low level
relationship prediction
Remote sensing
remote sensing image
scene graph generation
segmentation
Semantics
Sensors
Task analysis
Vegetation mapping
Visualization
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Remote sensing image analysis has drawn more attentions in the field of computer vision. At present, the methods commonly used in remote sensing image analysis are mainly at a low level semantically. The scene graph is an abstraction of objects and their relationships, which is a high level image understanding task. To fully comprehend the meanings of remote sensing images, in this paper, we propose a novel Segmentation-based model to generate Remote sensing image Scene Graphs (SRSG) . In the SRSG model, a more complete and accurate scene graph is generated with the segmentation results as inputs, while the shapes of objects are reasonably coded. The morphological features of object pairs are embedded together by different branches of the SRSG model and then mapped to semantic space to predicate their relationships. Furthermore, a new dataset for scene graph generation of remote sensing images, namely Segmentation results to Scene Graphs (S2SG) , is constructed based on pixel-level segmentation results. Experimental results demonstrate that the performance of the SRSG model is far superior to the previous methods in the task of generating remote sensing image scene graphs. The proposed SRSG model opens up new possibilities for remote sensing image analysis at a high level. Moreover, the S2SG dataset further allows for the evaluation of different approaches and is provided for the benefit of the research community.
ISSN:0196-2892
1558-0644
DOI:10.1109/TGRS.2022.3185678