RAPNet: Residual Atrous Pyramid Network for Importance-Aware Street Scene Parsing

Street Scene Parsing (SSP) is a fundamental and important step for autonomous driving and traffic scene understanding. Recently, Fully Convolutional Network (FCN) based methods have delivered expressive performances with the help of large-scale dense-labeling datasets. However, in urban traffic envi...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on image processing 2020-01, Vol.29, p.1-1
Main Authors: Zhang, Pingping, Liu, Wei, Lei, Yinjie, Wang, Hongyu, Lu, Huchuan
Format: Article
Language:English
Subjects:
Driving
fully convolutional network
importance-aware feature selection
Labeling
Labels
Machine learning
residual atrous spatial pyramid
Scene analysis
Semantics
Street scene parsing
Walkways
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Street Scene Parsing (SSP) is a fundamental and important step for autonomous driving and traffic scene understanding. Recently, Fully Convolutional Network (FCN) based methods have delivered expressive performances with the help of large-scale dense-labeling datasets. However, in urban traffic environments, not all the labels contribute equally for making the control decision. Certain labels such as pedestrian, car, bicyclist, road lane or sidewalk would be more important in comparison with labels for vegetation, sky or building. Based on this fact, in this paper we propose a novel deep learning framework, named Residual Atrous Pyramid Network (RAPNet), for importance-aware SSP. More specifically, to incorporate the importance of various object classes, we propose an Importance-Aware Feature Selection (IAFS) mechanism which automatically selects the important features for label predictions. The IAFS can operate in each convolutional block, and the semantic features with different importance are captured in different channels so that they are automatically assigned with corresponding weights. To enhance the labeling coherence, we also propose a Residual Atrous Spatial Pyramid (RASP) module to sequentially aggregate global-to-local context information in a residual refinement manner. Extensive experiments on two public benchmarks have shown that our approach achieves new state-of-the-art performances, and can consistently obtain more accurate results on the semantic classes with high importance levels.
ISSN:1057-7149
1941-0042
DOI:10.1109/TIP.2020.2978339