Video Extrapolation Using Neighboring Frames

• Published in: ACM transactions on graphics 2019, Vol.38 (3), p.1-13
• Main Authors: Lee, Sangwoo, Lee, Jungjin, Kim, Bumki, Kim, Kyehyun, Noh, Junyong
• Format: Article
• Language: English
• ISSN: 0730-0301; 1557-7368
• DOI: 10.1145/3196492

With the popularity of immersive display systems that fill the viewer’s field of view (FOV) entirely, demand for wide FOV content has increased. A video extrapolation technique based on reuse of existing videos is one of the most efficient ways to produce wide FOV content. Extrapolating a video poses a great challenge, however, due to the insufficient amount of cues and information that can be leveraged for the estimation of the extended region. This article introduces a novel framework that allows the extrapolation of an input video and consequently converts a conventional content into one with wide FOV. The key idea of the proposed approach is to integrate the information from all frames in the input video into each frame. Utilizing the information from all frames is crucial because it is very difficult to achieve the goal with a two-dimensional transformation based approach when parallax caused by camera motion is apparent. Warping guided by three-dimensnional scene points matches the viewpoints between the different frames. The matched frames are blended to create extended views. Various experiments demonstrate that the results of the proposed method are more visually plausible than those produced using state-of-the-art techniques.