Orthogonal Matching Pursuit for Text Classification

In text classification, the problem of overfitting arises due to the high dimensionality, making regularization essential. Although classic regularizers provide sparsity, they fail to return highly accurate models. On the contrary, state-of-the-art group-lasso regularizers provide better results at...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2018-10
Main Authors: Skianis, Konstantinos, Tziortziotis, Nikolaos, Vazirgiannis, Michalis
Format: Article
Language:English
Subjects:
Classification
Empirical analysis
Greedy algorithms
Matching
Regularization
Sparsity
Text editing
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In text classification, the problem of overfitting arises due to the high dimensionality, making regularization essential. Although classic regularizers provide sparsity, they fail to return highly accurate models. On the contrary, state-of-the-art group-lasso regularizers provide better results at the expense of low sparsity. In this paper, we apply a greedy variable selection algorithm, called Orthogonal Matching Pursuit, for the text classification task. We also extend standard group OMP by introducing overlapping Group OMP to handle overlapping groups of features. Empirical analysis verifies that both OMP and overlapping GOMP constitute powerful regularizers, able to produce effective and very sparse models. Code and data are available online: https://github.com/y3nk0/OMP-for-Text-Classification .
ISSN:2331-8422