Dimensionality-Driven Learning with Noisy Labels

Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such datasets, by investigating the dimensionality of the deep representation subspace o...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2018-07
Main Authors: Ma, Xingjun, Wang, Yisen, Houle, Michael E, Zhou, Shuo, Erfani, Sarah M, Shu-Tao, Xia, Wijewickrema, Sudanthi, Bailey, James
Format: Article
Language:English
Subjects:
Datasets
Labels
Learning
Neural networks
Subspaces
Training
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such datasets, by investigating the dimensionality of the deep representation subspace of training samples. We show that from a dimensionality perspective, DNNs exhibit quite distinctive learning styles when trained with clean labels versus when trained with a proportion of noisy labels. Based on this finding, we develop a new dimensionality-driven learning strategy, which monitors the dimensionality of subspaces during training and adapts the loss function accordingly. We empirically demonstrate that our approach is highly tolerant to significant proportions of noisy labels, and can effectively learn low-dimensional local subspaces that capture the data distribution.
ISSN:2331-8422