Spacetime Gaussian Grouping for 4D Object Segmentation

Novel-view synthesis of dynamic 3D scenes is an attractive but challenging problem. Many recent methods extend the 3D Gaussian Splatting model with temporal attributes in order to achieve high-quality and real-time rendered dynamic 3D scene representation. In order to have a more complete understand...

Full description

Saved in:
Bibliographic Details
Main Authors: Wei, Bangning, Maraval, Joshua, Besnard, Baptiste, Outtas, Meriem, Kpalma, Kidiyo, Ramin, Nicolas, Zhang, Lu
Format: Conference Proceeding
Language:English
Subjects:
4D object segmentation
Dynamic scene synthesis
Image reconstruction
Image segmentation
Object segmentation
Real-time systems
Solid modeling
Streaming media
Target tracking
Three-dimensional displays
Video sequences
Visualization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Novel-view synthesis of dynamic 3D scenes is an attractive but challenging problem. Many recent methods extend the 3D Gaussian Splatting model with temporal attributes in order to achieve high-quality and real-time rendered dynamic 3D scene representation. In order to have a more complete understanding of 4D scenes, we propose Spacetime Gaussian Grouping, which classifies and segments the objects spatially and temporally. The proposed method trains the Spacetime Gaussians (STG) model in conjunction with the multi-view video object segmentation masks corresponding to the input images. These masks are used to label the 3D Gaussians with instance identities. The identities of the Gaussians allow them to be grouped in 4D space, which has a positive impact on reconstruction. Furthermore, it allows the user to select and edit the objects in a spatio-temporal manner.
ISSN:2471-8963
DOI:10.1109/EUVIP61797.2024.10772975