Recognition of Indoor Scenes Using 3-D Scene Graphs

• Published in: IEEE transactions on geoscience and remote sensing 2024, Vol.62, p.1-16
• Main Authors: Yue, Han, Lehtola, Ville, Wu, Hangbin, Vosselman, George, Li, Jincheng, Liu, Chun
• Format: Article
• Language: English
• Subjects: Accuracy; Classification; Cloud computing; Convolution; Datasets; Deep learning; Feature extraction; Graph classification; Graphical representations; Graphs; indoor; Indoor environments; Instance segmentation; Point cloud compression; point clouds; Recognition; Scene analysis; scene graphs; scene recognition; Semantics; Structural members; Three-dimensional displays
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2024.3387556

Scene recognition is a fundamental task in 3-D scene understanding. It answers the question, "What is this place?" In an indoor environment, the answer can be an office, kitchen, lobby, and so on. As the number of point clouds increases, using embedded point information in scene recognition becomes computationally heavy to process. To achieve computational efficiency and accurate classification, our idea is to use an indoor scene graph that represents the 3-D spatial structures via object instances. The proposed method comprises two parts, namely: 1) construction of indoor scene graphs leveraging object instances and their spatial relationships and 2) classification of these graphs using a deep learning network. Specifically, each indoor scene is represented by a graph, where each node represents either a structural element (like a ceiling, a wall, or a floor) or a piece of furniture (like a chair or a table), and each edge encodes the spatial relationship between these elements. Then, these graphs are used as input for our proposed graph classification network to learn different scene representations. The public indoor dataset, ScanNet v2, with 625.53 million points, is selected to test our method. Experiments yield good results with up to 88.00% accuracy and 82.30% F1 score in the fixed validation dataset and 90.46% accuracy and 81.45% F1 score in the ten-fold cross-validation method; moreover, if some indoor objects cannot be successfully identified, the scene classification accuracy depends sublinearly on the rate of missing objects in the scene.