Transfer and share: semi-supervised learning from long-tailed data

Long-Tailed Semi-Supervised Learning (LTSSL) aims to learn from class-imbalanced data where only a few samples are annotated. Existing solutions typically require substantial cost to solve complex optimization problems, or class-balanced undersampling which can result in information loss. In this pa...

Full description

Saved in:
Bibliographic Details
Published in:Machine learning 2024-04, Vol.113 (4), p.1725-1742
Main Authors: Wei, Tong, Liu, Qian-Yu, Shi, Jiang-Xin, Tu, Wei-Wei, Guo, Lan-Zhe
Format: Article
Language:English
Subjects:
Accuracy
Artificial Intelligence
Computer Science
Control
Datasets
Entropy
Feature extraction
Labels
Machine Learning
Mechatronics
Methods
Natural Language Processing (NLP)
Neural networks
Robotics
Semi-supervised learning
Simulation and Modeling
Special Issue of the ACML 2022
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Long-Tailed Semi-Supervised Learning (LTSSL) aims to learn from class-imbalanced data where only a few samples are annotated. Existing solutions typically require substantial cost to solve complex optimization problems, or class-balanced undersampling which can result in information loss. In this paper, we present the TRAS (TRAnsfer and Share) to effectively utilize long-tailed semi-supervised data. TRAS transforms the imbalanced pseudo-label distribution of a traditional SSL model via a delicate function to enhance the supervisory signals for minority classes. It then transfers the distribution to a target model such that the minority class will receive significant attention. Interestingly, TRAS shows that more balanced pseudo-label distribution can substantially benefit minority-class training, instead of seeking to generate accurate pseudo-labels as in previous works. To simplify the approach, TRAS merges the training of the traditional SSL model and the target model into a single procedure by sharing the feature extractor, where both classifiers help improve the representation learning. According to extensive experiments, TRAS delivers much higher accuracy than state-of-the-art methods in the entire set of classes as well as minority classes. Code for TRAS is available at  https://github.com/Stomach-ache/TRAS .
ISSN:0885-6125
1573-0565
DOI:10.1007/s10994-022-06247-z