3D Room Layout Estimation From a Single RGB Image

• Published in: IEEE transactions on multimedia 2020-11, Vol.22 (11), p.3014-3024
• Main Authors: Yan, Chenggang, Shao, Biyao, Zhao, Hao, Ning, Ruixin, Zhang, Yongdong, Xu, Feng
• Format: Article
• Language: English
• Subjects: 2D topology; 3D topology; convolutional neural networks; Estimation; Image edge detection; Image reconstruction; Indoor scene; Layout; Layouts; Neural networks; nonlinear optimization; Optimization; Real estate; Scene analysis; Three-dimensional displays; Topology; Two dimensional displays
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2020.2967645

3D layout is crucial for scene understanding and reconstruction, and very useful in applications like real estate and furniture design. In this paper, we propose a fully automatic solution to estimate 3D layout of an indoor scene from a single 2D image. Our technique contains two key components. Firstly, we train a neural network that directly estimates room structure lines from the input image. Secondly, we propose a novel technique to automatically identify the layout topology of an input image, followed by a nonlinear optimization with equality constraints to estimate the final 3D layout of a scene. Based on our knowledge, this is the first fully automatic technique to achieve single image-based 3D layout estimation of an indoor scene. We evaluate our method on the public datasets LSUN, Hedau and 3DGP and the results show that the proposed method achieves accurate 3D layout reconstruction on various images with different layout topologies.