Video semantic segmentation with low latency

• Published in: Telkomnika 2024-10, Vol.22 (5), p.1147-1156
• Main Authors: Gowda, D V Channappa, Kanagavalli, R
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; Autonomous cars; Classification; Computer vision; Decision making; Deep learning; Flow nets; Graphics processing units; Human engineering; Human performance; Image segmentation; Machine learning; Network latency; Network topologies; Neural networks; Optical flow (image analysis); Semantic segmentation; Semantics; Topology
• ISSN: 1693-6930; 2302-9293
• DOI: 10.12928/TELKOMNIKA.v22i5.25157

Recent advances in computer vision and deep learning algorithms have yielded intriguing results. It can perform tasks previously requiring human eyes and brains. Semantic video segmentation for autonomous cars is difficult due to the high cost, low latency, and performance requirements of convolutional neural networks (CNNs). Deep learning architectures like SegNet and FlowNet 2.0 on the Cambridge-driving labeled video database (CamVid) dataset enable low-latency pixel-wise semantic segmentation of video features. Because it uses SegNet and FlowNet topologies, it is ideal for practical applications. The decision network chooses an optical flow or segmentation network for an image frame based on the expected confidence score. Combining this decision-making method with adaptive scheduling of the key frame approach can speed up the process. ResNet50 SegNet has a "54.27%" mean intersection over union (MIoU) and a "19.57" average FPS. In addition to decision network and adaptive key frame sequencing, FlowNet2.0 increased graphics processing unit (GPU) frame processing per second to "30.19" with a MIoU of "47.65%". The GPU is used "47.65%" of the time. This performance gain illustrates that the video semantic segmentation network is faster without sacrificing quality.