Bi-SSC: Geometric-Semantic Bidirectional Fusion for Camera-Based 3D Semantic Scene Completion

Camera-based Semantic Scene Completion (SSC) is to infer the full geometry of objects and scenes from only 2D images. The task is particularly challenging for those in-visible areas, due to the inherent occlusions and lighting ambiguity. Existing works ignore the information missing or ambiguous in...

Full description

Saved in:
Bibliographic Details
Main Authors: Xue, Yujie, Li, Ruihui, Wu, Fan, Tang, Zhuo, Li, Kenli, Duan, Mingxing
Format: Conference Proceeding
Language:English
Subjects:
Computer vision
Correlation
Geometry
Lighting
Pattern recognition
Semantic Scene Completion
Semantics
Three-dimensional displays
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Camera-based Semantic Scene Completion (SSC) is to infer the full geometry of objects and scenes from only 2D images. The task is particularly challenging for those in-visible areas, due to the inherent occlusions and lighting ambiguity. Existing works ignore the information missing or ambiguous in those shaded and occluded areas, resulting in distorted geometric prediction. To address this issue, we propose a novel method, Bi-SSC, bidirectional geomet-ric semantic fusion for camera-based 3D semantic scene completion. The key insight is to use the neighboring structure of objects in the image and the spatial differences from different perspectives to compensate for the lack of information in occluded areas. Specifically, we introduce a spatial sensory fusion module with multiple association attention to improve semantic correlation in geometric distributions. This module works within single view and across stereo views to achieve global spatial consistency. Experimental results demonstrate that Bi-SSC outperforms state-of-the-art camera-based methods on SemanticKITTI, particularly excelling in those invisible and shaded areas.
ISSN:2575-7075
DOI:10.1109/CVPR52733.2024.01902