Online multi-label dependency topic models for text classification

Multi-label text classification is an increasingly important field as large amounts of text data are available and extracting relevant information is important in many application contexts. Probabilistic generative models are the basis of a number of popular text mining methods such as Naive Bayes o...

Full description

Saved in:
Bibliographic Details
Published in:Machine learning 2018-05, Vol.107 (5), p.859-886
Main Authors: Burkhardt, Sophie, Kramer, Stefan
Format: Article
Language:English
Subjects:
Artificial Intelligence
Bayesian analysis
Classification
Computer Science
Control
Data mining
Dirichlet problem
Labels
Machine learning
Mechatronics
Natural Language Processing (NLP)
Robotics
Simulation and Modeling
Text editing
Texts
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Multi-label text classification is an increasingly important field as large amounts of text data are available and extracting relevant information is important in many application contexts. Probabilistic generative models are the basis of a number of popular text mining methods such as Naive Bayes or Latent Dirichlet Allocation. However, Bayesian models for multi-label text classification often are overly complicated to account for label dependencies and skewed label frequencies while at the same time preventing overfitting. To solve this problem we employ the same technique that contributed to the success of deep learning in recent years: greedy layer-wise training. Applying this technique in the supervised setting prevents overfitting and leads to better classification accuracy. The intuition behind this approach is to learn the labels first and subsequently add a more abstract layer to represent dependencies among the labels. This allows using a relatively simple hierarchical topic model which can easily be adapted to the online setting. We show that our method successfully models dependencies online for large-scale multi-label datasets with many labels and improves over the baseline method not modeling dependencies. The same strategy, layer-wise greedy training, also makes the batch variant competitive with existing more complex multi-label topic models.
ISSN:0885-6125
1573-0565
DOI:10.1007/s10994-017-5689-6