Reconstructing Interlaced High-Dynamic-Range Video Using Joint Learning

• Published in: IEEE transactions on image processing 2017-11, Vol.26 (11), p.5353-5366
• Main Authors: Inchang Choi, Seung-Hwan Baek, Kim, Min H.
• Format: Article
• Language: English
• Subjects: Cameras; deinterlacing; denoising; Dictionaries; Dynamic range; high-dynamic-range video; Image coding; Image reconstruction; Noise reduction
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2017.2731211

For extending the dynamic range of video, it is a common practice to capture multiple frames sequentially with different exposures and combine them to extend the dynamic range of each video frame. However, this approach results in typical ghosting artifacts due to fast and complex motion in nature. As an alternative, video imaging with interlaced exposures has been introduced to extend the dynamic range. However, the interlaced approach has been hindered by jaggy artifacts and sensor noise, leading to concerns over image quality. In this paper, we propose a data-driven approach for jointly solving two specific problems of deinterlacing and denoising that arise in interlaced video imaging with different exposures. First, we solve the deinterlacing problem using joint dictionary learning via sparse coding. Since partial information of detail in differently exposed rows is often available via interlacing, we make use of the information to reconstruct details of the extended dynamic range from the interlaced video input. Second, we jointly solve the denoising problem by tailoring sparse coding to better handle additive noise in low-/high-exposure rows, and also adopt multiscale homography flow to temporal sequences for denoising. We anticipate that the proposed method will allow for concurrent capture of higher dynamic range video frames without suffering from ghosting artifacts. We demonstrate the advantages of our interlaced video imaging compared with the state-of-the-art high-dynamic-range video methods.