Towards HDR and HFR Video from Rolling-Mixed-Bit Spikings

The spiking cameras offer the benefits of high dynamic range (HDR), high temporal resolution, and low data redundancy. However, reconstructing HDR videos in high-speed conditions using single-bit spikings presents challenges due to the limited bit depth. Increasing the bit depth of the spikings is a...

Full description

Saved in:
Bibliographic Details
Main Authors: Chang, Yakun, Xiaokaiti, Yeliduosi, Liu, Yujia, Fan, Bin, Huang, Zhaojun, Huang, Tiejun, Shi, Boxin
Format: Conference Proceeding
Language:English
Subjects:
Boosting
Cameras
Computational modeling
Computer vision
Data models
HDR video
high frame rate video
neuromorphic sensor
Pattern recognition
Redundancy
spiking camera
video reconstruction
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The spiking cameras offer the benefits of high dynamic range (HDR), high temporal resolution, and low data redundancy. However, reconstructing HDR videos in high-speed conditions using single-bit spikings presents challenges due to the limited bit depth. Increasing the bit depth of the spikings is advantageous for boosting HDR performance, but the readout efficiency will be decreased, which is unfavorable for achieving a high frame rate (HFR) video. To address these challenges, we propose a readout mechanism to obtain rolling-mixed-bit (RMB) spikings, which involves inter-leaving multi-bit spikings within the single-bit spikings in a rolling manner, thereby combining the characteristics of high bit depth and efficient readout. Furthermore, we introduce RMB-Net for reconstructing HDR and HFR videos. RMB-Net comprises a cross-bit attention block for fusing mixed-bit spikings and a cross-time attention block for achieving temporal fusion. Extensive experiments conducted on synthetic and real-synthetic data demonstrate the superiority of our method. For instance, pure 3 -bit spikings result in 3 times of data volume, whereas our method achieves comparable performance with less than 2% increase in data volume.
ISSN:2575-7075
DOI:10.1109/CVPR52733.2024.02373