A 0.5-V Real-Time Computational CMOS Image Sensor With Programmable Kernel for Feature Extraction

• Published in: IEEE journal of solid-state circuits 2021-05, Vol.56 (5), p.1588-1596
• Main Authors: Hsu, Tzu-Hsiang, Chen, Yi-Ren, Liu, Ren-Shuo, Lo, Chung-Chuan, Tang, Kea-Tiong, Chang, Meng-Fan, Hsieh, Chih-Cheng
• Format: Article
• Language: English
• Subjects: Analog to digital conversion; Analog to digital converters; Artificial intelligence; Artificial intelligence (AI); Artificial neural networks; Circuits; CMOS; computational CMOS image sensor (C²IS); Convolution; Feature extraction; Gesture recognition; Internet of Things; Kernel; Kernels; multiply–accumulate (MAC); Partial discharges; Pixels; Power consumption; Power management; processing-in-sensor (PIS); Prototypes; Pulse duration modulation; Pulse width modulation; pulsewidth modulation (PWM); Real time; Real-time systems; Sensor arrays; Sensors; Smart sensors; switch-current integration (SCI)
• ISSN: 0018-9200; 1558-173X
• DOI: 10.1109/JSSC.2020.3034192

As the growing demand on artificial intelligence (AI) Internet-of-Things (IoT) devices, smart vision sensors with energy-efficient computing capability are required. This article presents a low-power and low-voltage dual mode 0.5-V computational CMOS image sensor (C 2 IS) with array-parallel computing capability for feature extraction using convolution. In the feature extraction mode, by applying the pulsewidth modulation (PWM) pixel and switch-current integration (SCI) circuit, the in-sensor eight-directional matrix-parallel multiply-accumulate (MAC) operation is realized. Furthermore, the analog-domain convolution-on-readout (COR) operation, the programmable 3\times3 kernel with ±3-bit weights, and the tunable-resolution column-parallel analog-to-digital converter (ADC) (1-8 bit) are implemented to achieve the real-time feature extraction without using additional memory and sacrificing frame rate. In the image capturing mode, the sensor provides the linear-response 8-bit raw image data. The C 2 IS prototype has been fabricated in the TSMC 0.18- \mu \text{m} standard process technology and verified to demonstrate the raw and feature images at 480 frames/s with a power consumption of 77/ 117~\mu \text{W} and the resultant FoM of 9.8/14.8 pJ/pixel/frame, respectively. The prototype sensor is used as a real-time edge feature detection frond-end camera and accompanied with a simplified convolutional neural network (CNN) architecture to demonstrate the hand gesture recognition. The prototype system achieves more than 95% validation accuracy.