Semantic segmentation of point clouds of building interiors with deep learning: Augmenting training datasets with synthetic BIM-based point clouds

• Published in: Automation in construction 2020-05, Vol.113, p.103144, Article 103144
• Main Authors: Ma, Jong Won, Czerniawski, Thomas, Leite, Fernanda
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Building information model; Building information modeling; Building management systems; Datasets; Deep learning; Deep learning algorithm; Image segmentation; Neural networks; Point clouds; Semantic segmentation; Semantics; Synthetic dataset; Three dimensional models; Training
• ISSN: 0926-5805; 1872-7891
• DOI: 10.1016/j.autcon.2020.103144

This paper investigates the viability of using synthetic point clouds generated from building information models (BIMs) to train deep neural networks to perform semantic segmentation of point clouds of building interiors. In order to achieve these goals, this paper first presents a procedure for converting digital 3D BIMs into synthetic point clouds using three commercially available software systems. Then the generated synthetic point clouds are used to train a deep neural network. Semantic segmentation performance is compared for several models trained on: real point clouds, synthetic point clouds, and combinations of real and synthetic point clouds. A key finding is the 7.1% IOU boost in performance achieved when a small real point cloud dataset is augmented by synthetic point clouds for training, as compared to training the classifier on the real data alone. The experimental results confirmed the viability of using synthetic point clouds generated from building information models in combination with small datasets of real point clouds. This opens up the possibility of developing a segmentation model for building interiors that can be applied to as-built modeling of buildings that contain unseen indoor structures. •A methodology for generating synthetic point clouds from 3D BIM models•Semantic segmentation performance of synthetic point clouds using deep learning•Synthetic point clouds are effective when augmenting a small set of real point clouds.