Frequency-Based Matcher for Long-Tailed Semantic Segmentation

The successful application of semantic segmentation technology in the real world has been among the most exciting achievements in the computer vision community over the past decade. Although the long-tailed phenomenon has been investigated in many fields, e.g. , classification and object detection,...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on multimedia 2024, Vol.26, p.10395-10405
Main Authors: Li, Shan, Yang, Lu, Cao, Pu, Li, Liulei, Ma, Huadong
Format: Article
Language:English
Subjects:
Decoding
frequency-based matcher
Image segmentation
long-tailed learning
Semantic segmentation
Semantics
Tail
Task analysis
Transformers
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The successful application of semantic segmentation technology in the real world has been among the most exciting achievements in the computer vision community over the past decade. Although the long-tailed phenomenon has been investigated in many fields, e.g. , classification and object detection, it has not received enough attention in semantic segmentation and has become a nonnegligible obstacle to applying semantic segmentation technology in autonomous driving and virtual reality. Therefore, in this work, we focus on a relatively underexplored task setting, long-tailed semantic segmentation ( LTSS ). We first establish three representative datasets from different aspects, i.e., scene, object, and human. We further propose a dual-metric evaluation system and construct the LTSS benchmark to demonstrate the performance of semantic segmentation methods and long-tailed solutions. We also propose a transformer-based algorithm to improve LTSS, frequency-based matcher , which solves the oversuppression problem by one-to-many matching and automatically determines the number of matching queries for each class. Given the comprehensiveness of this work and the importance of the issues revealed, this work aims to promote the empirical study of semantic segmentation tasks.
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2024.3407679