Handheld snapshot multi-spectral camera at tens-of-megapixel resolution

Multi-spectral imaging is a fundamental tool characterizing the constituent energy of scene radiation. However, current multi-spectral video cameras cannot scale up beyond megapixel resolution due to optical constraints and the complexity of the reconstruction algorithms. To circumvent the above iss...

Full description

Saved in:
Bibliographic Details
Published in:Nature communications 2023-08, Vol.14 (1), p.5043-5043, Article 5043
Main Authors: Zhang, Weihang, Suo, Jinli, Dong, Kaiming, Li, Lianglong, Yuan, Xin, Pei, Chengquan, Dai, Qionghai
Format: Article
Language:English
Subjects:
639/624/1075/1083
639/624/1107/510
Algorithms
Camcorders
Cameras
Coding
Decoding
Design
Fiber optics
Humanities and Social Sciences
Image reconstruction
multidisciplinary
Scaling up
Science
Science (multidisciplinary)
Videography
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Multi-spectral imaging is a fundamental tool characterizing the constituent energy of scene radiation. However, current multi-spectral video cameras cannot scale up beyond megapixel resolution due to optical constraints and the complexity of the reconstruction algorithms. To circumvent the above issues, we propose a tens-of-megapixel handheld multi-spectral videography approach (THETA), with a proof-of-concept camera achieving 65-megapixel videography of 12 wavebands within visible light range. The high performance is brought by multiple designs: We propose an imaging scheme to fabricate a thin mask for encoding spatio-spectral data using a conventional film camera. Afterwards, a fiber optic plate is introduced for building a compact prototype supporting pixel-wise encoding with a large space-bandwidth product. Finally, a deep-network-based algorithm is adopted for large-scale multi-spectral data decoding, with the coding pattern specially designed to facilitate efficient coarse-to-fine model training. Experimentally, we demonstrate THETA’s advantageous and wide applications in outdoor imaging of large macroscopic scenes. Current multispectral video cameras can hardly scale up beyond megapixel resolution and do not apply to large-scale scenes. The authors design a tens-ofmegapixel handheld multi-spectral imaging scheme, build a setup achieving 65-megapixel videography of 12 wavebands, and show its wide applications.
ISSN:2041-1723
2041-1723
DOI:10.1038/s41467-023-40739-3