Simple Baseline Machine Learning Text Classifiers for Small Datasets

Text classification is important to better understand online media. A major problem for creating accurate text classifiers using machine learning is small training sets due to the cost of annotating them. On this basis, we investigated how SVM and NBSVM text classifiers should be designed to achieve...

Full description

Saved in:
Bibliographic Details
Published in:SN computer science 2021-05, Vol.2 (3), p.178, Article 178
Main Authors: Riekert, Martin, Riekert, Matthias, Klein, Achim
Format: Article
Language:English
Subjects:
Accuracy
Annotations
Classification
Classifiers
Computer Imaging
Computer Science
Computer Systems Organization and Communication Networks
Data Structures and Information Theory
Datasets
Design
Design factors
Dictionaries
Estimates
Factorial design
Feature selection
Information Systems and Communication Service
Learning curves
Machine learning
Original Research
Pattern Recognition and Graphics
Software Engineering/Programming and Operating Systems
Support vector machines
Text categorization
Vision
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Text classification is important to better understand online media. A major problem for creating accurate text classifiers using machine learning is small training sets due to the cost of annotating them. On this basis, we investigated how SVM and NBSVM text classifiers should be designed to achieve high accuracy and how the training sets should be sized to efficiently use annotation labor. We used a four-way repeated-measures full-factorial design of 32 design factor combinations. For each design factor combination 22 training set sizes were examined. These training sets were subsets of seven public text datasets. We study the statistical variance of accuracy estimates by randomly drawing new training sets, resulting in accuracy estimates for 98,560 different experimental runs. Our major contribution is a set of empirically evaluated guidelines for creating online media text classifiers using small training sets. We recommend uni- and bi-gram features as text representation, btc term weighting and a linear-kernel NBSVM. Our results suggest that high classification accuracy can be achieved using a manually annotated dataset of only 300 examples.
ISSN:2662-995X
2661-8907
DOI:10.1007/s42979-021-00480-4