PFPS: Polymerized Feature Panoptic Segmentation Based on Fully Convolutional Networks

• Published in: IEEE transactions on emerging topics in computational intelligence 2024-12, p.1-13
• Main Authors: Ji, Shucheng, Yuan, Xiaochen, Bao, Junqi, Liu, Tong, Lian, Yang, Huang, Guoheng, Zhong, Guo
• Format: Article
• Language: English
• Subjects: Convolution; Convolutional neural networks; deep learning; deep neural networks; Feature extraction; fully convolutional networks; Instance segmentation; Kernel; Location awareness; Panoptic segmentation; Pipelines; Polymers; Semantic segmentation; Semantics; supervised learning
• ISSN: 2471-285X
• DOI: 10.1109/TETCI.2024.3515004

Panoptic segmentation requires the prediction of a pixel-level mask with a category label in an image. In recent years, panoptic segmentation has been gaining more attention since it can help us understand objects and the environment in many fields, such as medical images, remote sensing images, and autonomous driving. However, existing panoptic segmentation methods are usually challenging for multi-scale object segmentation and boundary localization. In this paper, we propose a Polymerized Feature Panoptic Segmentation (PFPS) to enhance the network's feature representation ability by polymerizing the extracted stage features. Specifically, we propose a Generalization-Enhanced Stage Feature Generation Module (GSFGM) to extract and enhance the stage features. In the GSFGM, a novel Sampled and Concated Feature Generation (SCFG) is designed as an individual component, which polymerizes the convoluted backbone features to enhance multi-scale feature representation. Thereafter, we propose a Stage Feature Re-weight Module (SFRM) to ensure the network can learn efficient information from the massive channels. Moreover, we further propose a Unified Encoder Module (UEM) to provide spatial information and compress the high-dimensional features by coordinating convolution operations and channel attention. To demonstrate the superiority of the proposed PFPS, we conduct experiments on the COCO-2017 and the Cityscapes validation datasets. The experimental results indicate that the PFPS achieves a better performance in PQ of 43.0%, SQ of 80.4%, RQ of 51.9%, PQ th of 48.6%, SQ th of 82.6%, RQ th of 58.1%, PQ st of 34.6% on COCO-2017 validation dataset, while PQ of 61.7%, and PQ st of 67.9% on Cityscapes validation dataset.