Loading…
An Ultra-Low-Power Image Signal Processor for Hierarchical Image Recognition With Deep Neural Networks
We propose an ultra-low-power (ULP) image signal processor (ISP) that performs on-the-fly in-processing frame compression/decompression and hierarchical event recognition to exploit the temporal and spatial sparsity in an image sequence. This approach reduces energy consumption spent processing and...
Saved in:
| Published in: | IEEE journal of solid-state circuits 2021-04, Vol.56 (4), p.1071-1081 |
|---|---|
| Main Authors: | , , , , , , , , , , , , , , , |
| Format: | Article |
| Language: | English |
| Subjects: | |
| Citations: | Items that this one cites Items that cite this one |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| cited_by | cdi_FETCH-LOGICAL-c293t-97b3b1a4cae2160ee482e20756405f16df85b13936900d70403ee893e5ab11583 |
|---|---|
| cites | cdi_FETCH-LOGICAL-c293t-97b3b1a4cae2160ee482e20756405f16df85b13936900d70403ee893e5ab11583 |
| container_end_page | 1081 |
| container_issue | 4 |
| container_start_page | 1071 |
| container_title | IEEE journal of solid-state circuits |
| container_volume | 56 |
| creator | An, Hyochan Schiferl, Sam Venkatesan, Siddharth Wesley, Tim Zhang, Qirui Wang, Jingcheng Choo, Kyojin D. Liu, Shiyu Liu, Bowen Li, Ziyun Gong, Luyao Zhong, Hengfei Blaauw, David Dreslinski, Ronald Kim, Hun Seok Sylvester, Dennis |
| description | We propose an ultra-low-power (ULP) image signal processor (ISP) that performs on-the-fly in-processing frame compression/decompression and hierarchical event recognition to exploit the temporal and spatial sparsity in an image sequence. This approach reduces energy consumption spent processing and transmitting unimportant image data to achieve a 16 \times imaging system energy gain in an intruder detection scenario. The ISP was fabricated in 40-nm CMOS and consumes only 170 \mu \text{W} at 5 frames/s for neural network-based intruder detection and 192 \times compressed image recording. |
| doi_str_mv | 10.1109/JSSC.2020.3041858 |
| format | article |
| fullrecord | <record><control><sourceid>proquest_ieee_</sourceid><recordid>TN_cdi_proquest_journals_2506473324</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9293122</ieee_id><sourcerecordid>2506473324</sourcerecordid><originalsourceid>FETCH-LOGICAL-c293t-97b3b1a4cae2160ee482e20756405f16df85b13936900d70403ee893e5ab11583</originalsourceid><addsrcrecordid>eNo9kEtPwkAQgDdGExH9AcZLE8_FmX202yPBBxiCRCR6a7ZlCkXo4m4J8d9bUuJhMpnMN5OZj7FbhB4iJA-vs9mgx4FDT4BErfQZ66BSOsRYfJ2zDgDqMOEAl-zK-3VTSqmxw4p-Fcw3tTPh2B7CqT2QC0Zbs6RgVi4rswmmzubkvXVB0cSwJGdcvirzptVy75TbZVXWpa2Cz7JeBY9Eu2BCe9cgE6oP1n37a3ZRmI2nm1Pusvnz08dgGI7fXkaD_jjMeSLqMIkzkaGRuSGOERBJzYlDrCIJqsBoUWiVoUhElAAsYpAgiHQiSJkMUWnRZfft3p2zP3vydbq2e9f84VOuIJKxEFw2FLZU7qz3jop058qtcb8pQnrUmR51pked6UlnM3PXzpRE9M8nzdnIufgD1Ilvzg</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2506473324</pqid></control><display><type>article</type><title>An Ultra-Low-Power Image Signal Processor for Hierarchical Image Recognition With Deep Neural Networks</title><source>IEEE Electronic Library (IEL) Journals</source><creator>An, Hyochan ; Schiferl, Sam ; Venkatesan, Siddharth ; Wesley, Tim ; Zhang, Qirui ; Wang, Jingcheng ; Choo, Kyojin D. ; Liu, Shiyu ; Liu, Bowen ; Li, Ziyun ; Gong, Luyao ; Zhong, Hengfei ; Blaauw, David ; Dreslinski, Ronald ; Kim, Hun Seok ; Sylvester, Dennis</creator><creatorcontrib>An, Hyochan ; Schiferl, Sam ; Venkatesan, Siddharth ; Wesley, Tim ; Zhang, Qirui ; Wang, Jingcheng ; Choo, Kyojin D. ; Liu, Shiyu ; Liu, Bowen ; Li, Ziyun ; Gong, Luyao ; Zhong, Hengfei ; Blaauw, David ; Dreslinski, Ronald ; Kim, Hun Seok ; Sylvester, Dennis</creatorcontrib><description><![CDATA[We propose an ultra-low-power (ULP) image signal processor (ISP) that performs on-the-fly in-processing frame compression/decompression and hierarchical event recognition to exploit the temporal and spatial sparsity in an image sequence. This approach reduces energy consumption spent processing and transmitting unimportant image data to achieve a 16<inline-formula> <tex-math notation="LaTeX">\times </tex-math></inline-formula> imaging system energy gain in an intruder detection scenario. The ISP was fabricated in 40-nm CMOS and consumes only 170 <inline-formula> <tex-math notation="LaTeX">\mu \text{W} </tex-math></inline-formula> at 5 frames/s for neural network-based intruder detection and 192<inline-formula> <tex-math notation="LaTeX">\times </tex-math></inline-formula> compressed image recording.]]></description><identifier>ISSN: 0018-9200</identifier><identifier>EISSN: 1558-173X</identifier><identifier>DOI: 10.1109/JSSC.2020.3041858</identifier><identifier>CODEN: IJSCBC</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Artificial neural networks ; CMOS ; Deep neural network (DNN) ; Energy consumption ; energy-efficient processor ; Engines ; event recognition ; Face recognition ; Image coding ; Image compression ; image signal processor (ISP) ; Image transmission ; Imaging ; Microprocessors ; Neural networks ; Object recognition ; Power management ; Random access memory ; Signal processing ; Streaming media ; Transform coding</subject><ispartof>IEEE journal of solid-state circuits, 2021-04, Vol.56 (4), p.1071-1081</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c293t-97b3b1a4cae2160ee482e20756405f16df85b13936900d70403ee893e5ab11583</citedby><cites>FETCH-LOGICAL-c293t-97b3b1a4cae2160ee482e20756405f16df85b13936900d70403ee893e5ab11583</cites><orcidid>0000-0001-6744-7075 ; 0000-0001-8113-3558 ; 0000-0001-5831-9063 ; 0000-0002-6322-025X ; 0000-0001-6070-6310 ; 0000-0001-6046-6599 ; 0000-0001-8119-094X ; 0000-0003-2598-0458 ; 0000-0002-6658-5502</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9293122$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,27924,27925,54796</link.rule.ids></links><search><creatorcontrib>An, Hyochan</creatorcontrib><creatorcontrib>Schiferl, Sam</creatorcontrib><creatorcontrib>Venkatesan, Siddharth</creatorcontrib><creatorcontrib>Wesley, Tim</creatorcontrib><creatorcontrib>Zhang, Qirui</creatorcontrib><creatorcontrib>Wang, Jingcheng</creatorcontrib><creatorcontrib>Choo, Kyojin D.</creatorcontrib><creatorcontrib>Liu, Shiyu</creatorcontrib><creatorcontrib>Liu, Bowen</creatorcontrib><creatorcontrib>Li, Ziyun</creatorcontrib><creatorcontrib>Gong, Luyao</creatorcontrib><creatorcontrib>Zhong, Hengfei</creatorcontrib><creatorcontrib>Blaauw, David</creatorcontrib><creatorcontrib>Dreslinski, Ronald</creatorcontrib><creatorcontrib>Kim, Hun Seok</creatorcontrib><creatorcontrib>Sylvester, Dennis</creatorcontrib><title>An Ultra-Low-Power Image Signal Processor for Hierarchical Image Recognition With Deep Neural Networks</title><title>IEEE journal of solid-state circuits</title><addtitle>JSSC</addtitle><description><![CDATA[We propose an ultra-low-power (ULP) image signal processor (ISP) that performs on-the-fly in-processing frame compression/decompression and hierarchical event recognition to exploit the temporal and spatial sparsity in an image sequence. This approach reduces energy consumption spent processing and transmitting unimportant image data to achieve a 16<inline-formula> <tex-math notation="LaTeX">\times </tex-math></inline-formula> imaging system energy gain in an intruder detection scenario. The ISP was fabricated in 40-nm CMOS and consumes only 170 <inline-formula> <tex-math notation="LaTeX">\mu \text{W} </tex-math></inline-formula> at 5 frames/s for neural network-based intruder detection and 192<inline-formula> <tex-math notation="LaTeX">\times </tex-math></inline-formula> compressed image recording.]]></description><subject>Artificial neural networks</subject><subject>CMOS</subject><subject>Deep neural network (DNN)</subject><subject>Energy consumption</subject><subject>energy-efficient processor</subject><subject>Engines</subject><subject>event recognition</subject><subject>Face recognition</subject><subject>Image coding</subject><subject>Image compression</subject><subject>image signal processor (ISP)</subject><subject>Image transmission</subject><subject>Imaging</subject><subject>Microprocessors</subject><subject>Neural networks</subject><subject>Object recognition</subject><subject>Power management</subject><subject>Random access memory</subject><subject>Signal processing</subject><subject>Streaming media</subject><subject>Transform coding</subject><issn>0018-9200</issn><issn>1558-173X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><recordid>eNo9kEtPwkAQgDdGExH9AcZLE8_FmX202yPBBxiCRCR6a7ZlCkXo4m4J8d9bUuJhMpnMN5OZj7FbhB4iJA-vs9mgx4FDT4BErfQZ66BSOsRYfJ2zDgDqMOEAl-zK-3VTSqmxw4p-Fcw3tTPh2B7CqT2QC0Zbs6RgVi4rswmmzubkvXVB0cSwJGdcvirzptVy75TbZVXWpa2Cz7JeBY9Eu2BCe9cgE6oP1n37a3ZRmI2nm1Pusvnz08dgGI7fXkaD_jjMeSLqMIkzkaGRuSGOERBJzYlDrCIJqsBoUWiVoUhElAAsYpAgiHQiSJkMUWnRZfft3p2zP3vydbq2e9f84VOuIJKxEFw2FLZU7qz3jop058qtcb8pQnrUmR51pked6UlnM3PXzpRE9M8nzdnIufgD1Ilvzg</recordid><startdate>20210401</startdate><enddate>20210401</enddate><creator>An, Hyochan</creator><creator>Schiferl, Sam</creator><creator>Venkatesan, Siddharth</creator><creator>Wesley, Tim</creator><creator>Zhang, Qirui</creator><creator>Wang, Jingcheng</creator><creator>Choo, Kyojin D.</creator><creator>Liu, Shiyu</creator><creator>Liu, Bowen</creator><creator>Li, Ziyun</creator><creator>Gong, Luyao</creator><creator>Zhong, Hengfei</creator><creator>Blaauw, David</creator><creator>Dreslinski, Ronald</creator><creator>Kim, Hun Seok</creator><creator>Sylvester, Dennis</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SP</scope><scope>8FD</scope><scope>L7M</scope><orcidid>https://orcid.org/0000-0001-6744-7075</orcidid><orcidid>https://orcid.org/0000-0001-8113-3558</orcidid><orcidid>https://orcid.org/0000-0001-5831-9063</orcidid><orcidid>https://orcid.org/0000-0002-6322-025X</orcidid><orcidid>https://orcid.org/0000-0001-6070-6310</orcidid><orcidid>https://orcid.org/0000-0001-6046-6599</orcidid><orcidid>https://orcid.org/0000-0001-8119-094X</orcidid><orcidid>https://orcid.org/0000-0003-2598-0458</orcidid><orcidid>https://orcid.org/0000-0002-6658-5502</orcidid></search><sort><creationdate>20210401</creationdate><title>An Ultra-Low-Power Image Signal Processor for Hierarchical Image Recognition With Deep Neural Networks</title><author>An, Hyochan ; Schiferl, Sam ; Venkatesan, Siddharth ; Wesley, Tim ; Zhang, Qirui ; Wang, Jingcheng ; Choo, Kyojin D. ; Liu, Shiyu ; Liu, Bowen ; Li, Ziyun ; Gong, Luyao ; Zhong, Hengfei ; Blaauw, David ; Dreslinski, Ronald ; Kim, Hun Seok ; Sylvester, Dennis</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c293t-97b3b1a4cae2160ee482e20756405f16df85b13936900d70403ee893e5ab11583</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Artificial neural networks</topic><topic>CMOS</topic><topic>Deep neural network (DNN)</topic><topic>Energy consumption</topic><topic>energy-efficient processor</topic><topic>Engines</topic><topic>event recognition</topic><topic>Face recognition</topic><topic>Image coding</topic><topic>Image compression</topic><topic>image signal processor (ISP)</topic><topic>Image transmission</topic><topic>Imaging</topic><topic>Microprocessors</topic><topic>Neural networks</topic><topic>Object recognition</topic><topic>Power management</topic><topic>Random access memory</topic><topic>Signal processing</topic><topic>Streaming media</topic><topic>Transform coding</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>An, Hyochan</creatorcontrib><creatorcontrib>Schiferl, Sam</creatorcontrib><creatorcontrib>Venkatesan, Siddharth</creatorcontrib><creatorcontrib>Wesley, Tim</creatorcontrib><creatorcontrib>Zhang, Qirui</creatorcontrib><creatorcontrib>Wang, Jingcheng</creatorcontrib><creatorcontrib>Choo, Kyojin D.</creatorcontrib><creatorcontrib>Liu, Shiyu</creatorcontrib><creatorcontrib>Liu, Bowen</creatorcontrib><creatorcontrib>Li, Ziyun</creatorcontrib><creatorcontrib>Gong, Luyao</creatorcontrib><creatorcontrib>Zhong, Hengfei</creatorcontrib><creatorcontrib>Blaauw, David</creatorcontrib><creatorcontrib>Dreslinski, Ronald</creatorcontrib><creatorcontrib>Kim, Hun Seok</creatorcontrib><creatorcontrib>Sylvester, Dennis</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Xplore</collection><collection>CrossRef</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>Advanced Technologies Database with Aerospace</collection><jtitle>IEEE journal of solid-state circuits</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>An, Hyochan</au><au>Schiferl, Sam</au><au>Venkatesan, Siddharth</au><au>Wesley, Tim</au><au>Zhang, Qirui</au><au>Wang, Jingcheng</au><au>Choo, Kyojin D.</au><au>Liu, Shiyu</au><au>Liu, Bowen</au><au>Li, Ziyun</au><au>Gong, Luyao</au><au>Zhong, Hengfei</au><au>Blaauw, David</au><au>Dreslinski, Ronald</au><au>Kim, Hun Seok</au><au>Sylvester, Dennis</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>An Ultra-Low-Power Image Signal Processor for Hierarchical Image Recognition With Deep Neural Networks</atitle><jtitle>IEEE journal of solid-state circuits</jtitle><stitle>JSSC</stitle><date>2021-04-01</date><risdate>2021</risdate><volume>56</volume><issue>4</issue><spage>1071</spage><epage>1081</epage><pages>1071-1081</pages><issn>0018-9200</issn><eissn>1558-173X</eissn><coden>IJSCBC</coden><abstract><![CDATA[We propose an ultra-low-power (ULP) image signal processor (ISP) that performs on-the-fly in-processing frame compression/decompression and hierarchical event recognition to exploit the temporal and spatial sparsity in an image sequence. This approach reduces energy consumption spent processing and transmitting unimportant image data to achieve a 16<inline-formula> <tex-math notation="LaTeX">\times </tex-math></inline-formula> imaging system energy gain in an intruder detection scenario. The ISP was fabricated in 40-nm CMOS and consumes only 170 <inline-formula> <tex-math notation="LaTeX">\mu \text{W} </tex-math></inline-formula> at 5 frames/s for neural network-based intruder detection and 192<inline-formula> <tex-math notation="LaTeX">\times </tex-math></inline-formula> compressed image recording.]]></abstract><cop>New York</cop><pub>IEEE</pub><doi>10.1109/JSSC.2020.3041858</doi><tpages>11</tpages><orcidid>https://orcid.org/0000-0001-6744-7075</orcidid><orcidid>https://orcid.org/0000-0001-8113-3558</orcidid><orcidid>https://orcid.org/0000-0001-5831-9063</orcidid><orcidid>https://orcid.org/0000-0002-6322-025X</orcidid><orcidid>https://orcid.org/0000-0001-6070-6310</orcidid><orcidid>https://orcid.org/0000-0001-6046-6599</orcidid><orcidid>https://orcid.org/0000-0001-8119-094X</orcidid><orcidid>https://orcid.org/0000-0003-2598-0458</orcidid><orcidid>https://orcid.org/0000-0002-6658-5502</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0018-9200 |
| ispartof | IEEE journal of solid-state circuits, 2021-04, Vol.56 (4), p.1071-1081 |
| issn | 0018-9200 1558-173X |
| language | eng |
| recordid | cdi_proquest_journals_2506473324 |
| source | IEEE Electronic Library (IEL) Journals |
| subjects | Artificial neural networks CMOS Deep neural network (DNN) Energy consumption energy-efficient processor Engines event recognition Face recognition Image coding Image compression image signal processor (ISP) Image transmission Imaging Microprocessors Neural networks Object recognition Power management Random access memory Signal processing Streaming media Transform coding |
| title | An Ultra-Low-Power Image Signal Processor for Hierarchical Image Recognition With Deep Neural Networks |
| url | http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T20%3A27%3A24IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_ieee_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=An%20Ultra-Low-Power%20Image%20Signal%20Processor%20for%20Hierarchical%20Image%20Recognition%20With%20Deep%20Neural%20Networks&rft.jtitle=IEEE%20journal%20of%20solid-state%20circuits&rft.au=An,%20Hyochan&rft.date=2021-04-01&rft.volume=56&rft.issue=4&rft.spage=1071&rft.epage=1081&rft.pages=1071-1081&rft.issn=0018-9200&rft.eissn=1558-173X&rft.coden=IJSCBC&rft_id=info:doi/10.1109/JSSC.2020.3041858&rft_dat=%3Cproquest_ieee_%3E2506473324%3C/proquest_ieee_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c293t-97b3b1a4cae2160ee482e20756405f16df85b13936900d70403ee893e5ab11583%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2506473324&rft_id=info:pmid/&rft_ieee_id=9293122&rfr_iscdi=true |