A 1920 [Formula Omitted] 1080 25-Frames/s 2.4-TOPS/W Low-Power 6-D Vision Processor for Unified Optical Flow and Stereo Depth With Semi-Global Matching

• Published in: IEEE journal of solid-state circuits 2019-04, Vol.54 (4), p.1048
• Main Authors: Li, Ziyun, Wang, Jingcheng, Sylvester, Dennis, Blaauw, David, Kim, Hun Seok
• Format: Article
• Language: English
• Subjects: Algorithms; Circuit design; CMOS; Coalescing; Computer terminals; Dependence; Frames (data processing); Matching; Microprocessors; Motion perception; Optical flow (image analysis); Parallel processing; Pedestrians; Three dimensional motion; Vision
• ISSN: 0018-9200; 1558-173X
• DOI: 10.1109/JSSC.2018.2885559

This paper presents a unified 6-D vision processor that enables dense real-time 3-D depth and 3-D motion perception at full-high-definition ([Formula Omitted], FHD) resolution. The proposed design implements a neighbor-guided semi-global matching (NG-SGM) algorithm to unify the stereo depth and optical flow matching problem and to reduce computation by 98% compared with the original SGM. We introduce a new custom-designed, high-bandwidth coalescing crossbar circuit that automatically coalesces redundant memory accesses to mitigate the highly irregular memory accesses observed in NG-SGM. The proposed 6-D vision processor also maximizes on-chip memory reuse by using 64 on-chip rotating image buffers that cover a wide optical flow and depth disparity search range of 176 pixels per dimension. The processor implements massive parallel processing with 576 compute units that are deeply pipelined with a dependency-resolving skewed-diagonal scan to hide the dynamic and variable dependency in the pipeline. The fabricated processor performs dense NG-SGM at 25 frames/s for optical flow or 30 frames/s for stereo depth at FHD resolution while consuming only 760 mW in 28-nm CMOS.