A 65 nm 2.02 mW 50 Mbps Direct Analog to MJPEG Converter for Video Sensor Nodes using low-noise Switched Capacitor MAC-Quantizer with automatic calibration and Sparsity-aware ADC

• Main Authors: Gaurav Kumar, K, Barik, Gourab, Chatterjee, Baibhab, Bose, Sumon, Maity, Shovan, Sen, Shreyas
• Format: Conference Proceeding
• Language: English
• Subjects: Capacitors; Image coding; Robot vision systems; Streaming media; Surveillance; Switches; Wireless communication
• ISSN: 2152-3630
• DOI: 10.1109/CICC57935.2023.10121318

Recent expansions in camera deployments for applications like surveillance, healthcare, autonomous vehicles, robots necessitate low-power realizations even with large data volume. Standard digital video cameras generate \sim Gbps data that are usually compressed to 10's of Mbps, often in the camera (e.g. Omnivision 5640) using compression schemes, like lossy MJPEG before high energy/bit wireless communication. For instance (Fig. 1), a 2 \mathrm{~K} RGB 8 b video at 30 fps will generate \sim 1.5 \mathrm{Gbps} that is reduced to \sim 75 \mathrm{Mbps} (MJPEG: per-frame compression) or to \sim 6 \mathrm{Mbps} (H.264: both per-frame and inter-frame compression). While H.264 enables lower data rate, it suffers from complex decoding (incurs high latency), and poor error tolerance, thus MJPEG is preferred for latency-sensitive error-prone communication of compressed video with real-time inference needs. However, such in-camera ADC with Digital compression suffers from 1) high data volume (Gbps) generated, consuming high ADC energy; 2) high compute energy required by the in-sensor digital accelerator to compress the data, increasing the power consumption of the processing modules in these nodes to \sim 10 's mW. This work draws on the recent progress of mixed-signal in-sensor computing to address the above challenges in these high-power and high datavolume nodes and presents a "direct Analog to MJPEG Conversion (d-AJC)" technique that (1) compresses the analog pixel values from imagers in the mixed-signal domain using Switched Capacitor (SC)-based multiply and accumulate (MAC) units, and Quantizer, before (2) digitizing them with an event-driven sparsity aware ADC with run-length encoding, effectively compressing Gbps data in SC domain with 10's of Mbps ADC, leading to an order of magnitude system power benefit. With gaining prominence of low-power analog camera sensors (e.g., 25mW Omnivision 6946), along with the development of low-cost processing schemes, d-AJC, it is envisioned to solve issue of high-power imager nodes.