Loading…

Through the Curved Cover: Synthesizing Cover Aberrated Scenes with Refractive Field

Recent extended reality headsets and field robots have adopted covers to protect the front-facing cameras from environmental hazards and falls. The surface irregularities on the cover can lead to optical aberrations like blurring and non-parametric distortions. Novel view synthesis methods like NeRF...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-11
Main Authors: Xie, Liuyue, Guo, Jiancong, Jeni, Laszlo A, Jia, Zhiheng, Li, Mingyang, Zhou, Yunwen, Guo, Chao
Format: Article
Language:English
Subjects:
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Xie, Liuyue
Guo, Jiancong
Jeni, Laszlo A
Jia, Zhiheng
Li, Mingyang
Zhou, Yunwen
Guo, Chao
description Recent extended reality headsets and field robots have adopted covers to protect the front-facing cameras from environmental hazards and falls. The surface irregularities on the cover can lead to optical aberrations like blurring and non-parametric distortions. Novel view synthesis methods like NeRF and 3D Gaussian Splatting are ill-equipped to synthesize from sequences with optical aberrations. To address this challenge, we introduce SynthCover to enable novel view synthesis through protective covers for downstream extended reality applications. SynthCover employs a Refractive Field that estimates the cover's geometry, enabling precise analytical calculation of refracted rays. Experiments on synthetic and real-world scenes demonstrate our method's ability to accurately model scenes viewed through protective covers, achieving a significant improvement in rendering quality compared to prior methods. We also show that the model can adjust well to various cover geometries with synthetic sequences captured with covers of different surface curvatures. To motivate further studies on this problem, we provide the benchmarked dataset containing real and synthetic walkable scenes captured with protective cover optical aberrations.
format article
fullrecord <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_3127414531</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3127414531</sourcerecordid><originalsourceid>FETCH-proquest_journals_31274145313</originalsourceid><addsrcrecordid>eNqNis0KgkAYAJcgSMp3-KCz4O5qRreQpHN6D9NPd0V2a3-MevqEeoBOAzOzIAHjnEb7hLEVCa0d4jhmu4ylKQ9IWQmjfS_ACYTcmwlbyPWE5gDlS83SyrdU_dfB8YbG1G5-ygYVWnhKJ-CCnakbJyeEQuLYbsiyq0eL4Y9rsi1OVX6O7kY_PFp3HbQ3ak5XTlmW0CTllP93fQATJT--</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3127414531</pqid></control><display><type>article</type><title>Through the Curved Cover: Synthesizing Cover Aberrated Scenes with Refractive Field</title><source>Publicly Available Content (ProQuest)</source><creator>Xie, Liuyue ; Guo, Jiancong ; Jeni, Laszlo A ; Jia, Zhiheng ; Li, Mingyang ; Zhou, Yunwen ; Guo, Chao</creator><creatorcontrib>Xie, Liuyue ; Guo, Jiancong ; Jeni, Laszlo A ; Jia, Zhiheng ; Li, Mingyang ; Zhou, Yunwen ; Guo, Chao</creatorcontrib><description>Recent extended reality headsets and field robots have adopted covers to protect the front-facing cameras from environmental hazards and falls. The surface irregularities on the cover can lead to optical aberrations like blurring and non-parametric distortions. Novel view synthesis methods like NeRF and 3D Gaussian Splatting are ill-equipped to synthesize from sequences with optical aberrations. To address this challenge, we introduce SynthCover to enable novel view synthesis through protective covers for downstream extended reality applications. SynthCover employs a Refractive Field that estimates the cover's geometry, enabling precise analytical calculation of refracted rays. Experiments on synthetic and real-world scenes demonstrate our method's ability to accurately model scenes viewed through protective covers, achieving a significant improvement in rendering quality compared to prior methods. We also show that the model can adjust well to various cover geometries with synthetic sequences captured with covers of different surface curvatures. To motivate further studies on this problem, we provide the benchmarked dataset containing real and synthetic walkable scenes captured with protective cover optical aberrations.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Aberration ; Refracted waves ; Synthesis</subject><ispartof>arXiv.org, 2024-11</ispartof><rights>2024. This work is published under http://creativecommons.org/licenses/by-nc-nd/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/3127414531?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>780,784,25753,37012,44590</link.rule.ids></links><search><creatorcontrib>Xie, Liuyue</creatorcontrib><creatorcontrib>Guo, Jiancong</creatorcontrib><creatorcontrib>Jeni, Laszlo A</creatorcontrib><creatorcontrib>Jia, Zhiheng</creatorcontrib><creatorcontrib>Li, Mingyang</creatorcontrib><creatorcontrib>Zhou, Yunwen</creatorcontrib><creatorcontrib>Guo, Chao</creatorcontrib><title>Through the Curved Cover: Synthesizing Cover Aberrated Scenes with Refractive Field</title><title>arXiv.org</title><description>Recent extended reality headsets and field robots have adopted covers to protect the front-facing cameras from environmental hazards and falls. The surface irregularities on the cover can lead to optical aberrations like blurring and non-parametric distortions. Novel view synthesis methods like NeRF and 3D Gaussian Splatting are ill-equipped to synthesize from sequences with optical aberrations. To address this challenge, we introduce SynthCover to enable novel view synthesis through protective covers for downstream extended reality applications. SynthCover employs a Refractive Field that estimates the cover's geometry, enabling precise analytical calculation of refracted rays. Experiments on synthetic and real-world scenes demonstrate our method's ability to accurately model scenes viewed through protective covers, achieving a significant improvement in rendering quality compared to prior methods. We also show that the model can adjust well to various cover geometries with synthetic sequences captured with covers of different surface curvatures. To motivate further studies on this problem, we provide the benchmarked dataset containing real and synthetic walkable scenes captured with protective cover optical aberrations.</description><subject>Aberration</subject><subject>Refracted waves</subject><subject>Synthesis</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><recordid>eNqNis0KgkAYAJcgSMp3-KCz4O5qRreQpHN6D9NPd0V2a3-MevqEeoBOAzOzIAHjnEb7hLEVCa0d4jhmu4ylKQ9IWQmjfS_ACYTcmwlbyPWE5gDlS83SyrdU_dfB8YbG1G5-ygYVWnhKJ-CCnakbJyeEQuLYbsiyq0eL4Y9rsi1OVX6O7kY_PFp3HbQ3ak5XTlmW0CTllP93fQATJT--</recordid><startdate>20241110</startdate><enddate>20241110</enddate><creator>Xie, Liuyue</creator><creator>Guo, Jiancong</creator><creator>Jeni, Laszlo A</creator><creator>Jia, Zhiheng</creator><creator>Li, Mingyang</creator><creator>Zhou, Yunwen</creator><creator>Guo, Chao</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20241110</creationdate><title>Through the Curved Cover: Synthesizing Cover Aberrated Scenes with Refractive Field</title><author>Xie, Liuyue ; Guo, Jiancong ; Jeni, Laszlo A ; Jia, Zhiheng ; Li, Mingyang ; Zhou, Yunwen ; Guo, Chao</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_31274145313</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Aberration</topic><topic>Refracted waves</topic><topic>Synthesis</topic><toplevel>online_resources</toplevel><creatorcontrib>Xie, Liuyue</creatorcontrib><creatorcontrib>Guo, Jiancong</creatorcontrib><creatorcontrib>Jeni, Laszlo A</creatorcontrib><creatorcontrib>Jia, Zhiheng</creatorcontrib><creatorcontrib>Li, Mingyang</creatorcontrib><creatorcontrib>Zhou, Yunwen</creatorcontrib><creatorcontrib>Guo, Chao</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content (ProQuest)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Xie, Liuyue</au><au>Guo, Jiancong</au><au>Jeni, Laszlo A</au><au>Jia, Zhiheng</au><au>Li, Mingyang</au><au>Zhou, Yunwen</au><au>Guo, Chao</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>Through the Curved Cover: Synthesizing Cover Aberrated Scenes with Refractive Field</atitle><jtitle>arXiv.org</jtitle><date>2024-11-10</date><risdate>2024</risdate><eissn>2331-8422</eissn><abstract>Recent extended reality headsets and field robots have adopted covers to protect the front-facing cameras from environmental hazards and falls. The surface irregularities on the cover can lead to optical aberrations like blurring and non-parametric distortions. Novel view synthesis methods like NeRF and 3D Gaussian Splatting are ill-equipped to synthesize from sequences with optical aberrations. To address this challenge, we introduce SynthCover to enable novel view synthesis through protective covers for downstream extended reality applications. SynthCover employs a Refractive Field that estimates the cover's geometry, enabling precise analytical calculation of refracted rays. Experiments on synthetic and real-world scenes demonstrate our method's ability to accurately model scenes viewed through protective covers, achieving a significant improvement in rendering quality compared to prior methods. We also show that the model can adjust well to various cover geometries with synthetic sequences captured with covers of different surface curvatures. To motivate further studies on this problem, we provide the benchmarked dataset containing real and synthetic walkable scenes captured with protective cover optical aberrations.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2024-11
issn 2331-8422
language eng
recordid cdi_proquest_journals_3127414531
source Publicly Available Content (ProQuest)
subjects Aberration
Refracted waves
Synthesis
title Through the Curved Cover: Synthesizing Cover Aberrated Scenes with Refractive Field
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T22%3A13%3A52IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=Through%20the%20Curved%20Cover:%20Synthesizing%20Cover%20Aberrated%20Scenes%20with%20Refractive%20Field&rft.jtitle=arXiv.org&rft.au=Xie,%20Liuyue&rft.date=2024-11-10&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E3127414531%3C/proquest%3E%3Cgrp_id%3Ecdi_FETCH-proquest_journals_31274145313%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=3127414531&rft_id=info:pmid/&rfr_iscdi=true