An Overlay Accelerator of DeepLab CNN for Spacecraft Image Segmentation on FPGA

• Published in: Remote sensing (Basel, Switzerland) Switzerland), 2024-03, Vol.16 (5), p.894
• Main Authors: Guo, Zibo, Liu, Kai, Liu, Wei, Sun, Xiaoyao, Ding, Chongyang, Li, Shangrong
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Artificial neural networks; Co-design; Computational efficiency; Computer applications; Computing time; Data processing; Data transfer (computers); Design; Digital integrated circuits; Efficiency; field programmable gate array (FPGA); Field programmable gate arrays; Hardware; Image processing; Image segmentation; image semantic segmentation; instruction set architecture (ISA); Memory management; Neural networks; Performance enhancement; Semantics; Semiconductor industry; Space ships; Space vehicles; Spacecraft; spacecraft component images; Spacecraft components; Tiling
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs16050894

Due to the absence of communication and coordination with external spacecraft, non-cooperative spacecraft present challenges for the servicing spacecraft in acquiring information about their pose and location. The accurate segmentation of non-cooperative spacecraft components in images is a crucial step in autonomously sensing the pose of non-cooperative spacecraft. This paper presents a novel overlay accelerator of DeepLab Convolutional Neural Networks (CNNs) for spacecraft image segmentation on a FPGA. First, several software–hardware co-design aspects are investigated: (1) A CNNs-domain COD instruction set (Control, Operation, Data Transfer) is presented based on a Load–Store architecture to enable the implementation of accelerator overlays. (2) An RTL-based prototype accelerator is developed for the COD instruction set. The accelerator incorporates dedicated units for instruction decoding and dispatch, scheduling, memory management, and operation execution. (3) A compiler is designed that leverages tiling and operation fusion techniques to optimize the execution of CNNs, generating binary instructions for the optimized operations. Our accelerator is implemented on a Xilinx Virtex-7 XC7VX690T FPGA at 200 MHz. Experiments demonstrate that with INT16 quantization our accelerator achieves an accuracy (mIoU) of 77.84%, experiencing only a 0.2% degradation compared to that of the original fully precision model, in accelerating the segmentation model of DeepLabv3+ ResNet18 on the spacecraft component images (SCIs) dataset. The accelerator boasts a performance of 184.19 GOPS/s and a computational efficiency (Runtime Throughput/Theoretical Roof Throughput) of 88.72%. Compared to previous work, our accelerator improves performance by 1.5× and computational efficiency by 43.93%, all while consuming similar hardware resources. Additionally, in terms of instruction encoding, our instructions reduce the size by 1.5× to 49× when compiling the same model compared to previous work.