Separating the Wheat from the Chaff: Applications of Automated Document Classification Using Support Vector Machines

Due in large part to the proliferation of digitized text, much of it available for little or no cost from the Internet, political science research has experienced a substantial increase in the number of data sets and large-n research initiatives. As the ability to collect detailed information on eve...

Full description

Saved in:
Bibliographic Details
Published in:Political analysis 2014, Vol.22 (2), p.224-242
Main Authors: D'Orazio, Vito, Landis, Steven T., Palmer, Glenn, Schrodt, Philip
Format: Article
Language:English
Subjects:
Archives
Chaff
Classification
Data collection
Datasets
Information classification
Information relevance
Information retrieval
Internet
Machine learning
Political science
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Due in large part to the proliferation of digitized text, much of it available for little or no cost from the Internet, political science research has experienced a substantial increase in the number of data sets and large-n research initiatives. As the ability to collect detailed information on events of interest expands, so does the need to efficiently sort through the volumes of available information. Automated document classification presents a particularly attractive methodology for accomplishing this task. It is efficient, widely applicable to a variety of data collection efforts, and considerably flexible in tailoring its application for specific research needs. This article offers a holistic review of the application of automated document classification for data collection in political science research by discussing the process in its entirety. We argue that the application of a two-stage support vector machine (SVM) classification process offers advantages over other well-known alternatives, due to the nature of SVMs being a discriminative classifier and having the ability to effectively address two primary attributes of textual data: high dimensionality and extreme sparseness. Evidence for this claim is presented through a discussion of the efficiency gains derived from using automated document classification on the Militarized Interstate Dispute 4 (MID4) data collection project.
ISSN:1047-1987
1476-4989
DOI:10.1093/pan/mpt030