Emptying, refurnishing, and relighting indoor spaces

• Published in: ACM transactions on graphics 2016-11, Vol.35 (6), p.1-14
• Main Authors: Zhang, Edward, Cohen, Michael F., Curless, Brian
• Format: Article
• Language: English
• Subjects: Dynamical systems; Emptying; Furniture; Illumination; Lighting; Purchasing; Rendering
• ISSN: 0730-0301; 1557-7368
• DOI: 10.1145/2980179.2982432

Visualizing changes to indoor scenes is important for many applications. When looking for a new place to live, we want to see how the interior looks not with the current inhabitant's belongings, but with our own furniture. Before purchasing a new sofa, we want to visualize how it would look in our living room. In this paper, we present a system that takes an RGBD scan of an indoor scene and produces a scene model of the empty room, including light emitters, materials, and the geometry of the non-cluttered room. Our system enables realistic rendering not only of the empty room under the original lighting conditions, but also with various scene edits, including adding furniture, changing the material properties of the walls, and relighting. These types of scene edits enable many mixed reality applications in areas such as real estate, furniture retail, and interior design. Our system contains two novel technical contributions: a 3D radiometric calibration process that recovers the appearance of the scene in high dynamic range, and a global-illumination-aware inverse rendering framework that simultaneously recovers reflectance properties of scene surfaces and lighting properties for several light source types, including generalized point and line lights.