Language morphology offset: Text classification on a Croatian–English parallel corpus

We investigate how, and to what extent, morphological complexity of the language influences text classification using support vector machines (SVM). The Croatian–English parallel corpus provides the basis for direct comparison of two languages of radically different morphological complexity. We quan...

Full description

Saved in:
Bibliographic Details
Published in:Information processing & management 2008, Vol.44 (1), p.325-339
Main Authors: Malenica, M., Šmuc, T., Šnajder, J., Dalbelo Bašić, B.
Format: Article
Language:English
Subjects:
Classification
Comparative analysis
Computerized information retrieval
Content analysis
Croatian
English
Exact sciences and technology
Feature selection
Indexing. Classification. Abstracting
Indexing. Classification. Abstracting. Syntheses
Information and communication sciences
Information and document structure and analysis
Information processing
Information processing and retrieval
Information science. Documentation
Language
Languages
Lemmatization
Morphological normalisation
Morphology
Sciences and techniques of general use
Stemming
Studies
Support vector machines
SVM
Text categorization
Text classification
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We investigate how, and to what extent, morphological complexity of the language influences text classification using support vector machines (SVM). The Croatian–English parallel corpus provides the basis for direct comparison of two languages of radically different morphological complexity. We quantified, compared, and statistically tested the effects of morphological normalisation on SVM classifier performance based on a series of parallel experiments on both languages, carried over a large scale of different feature subset sizes obtained by different feature selection methods, and applying different levels of morphological normalisation. We also quantified the trade-off between feature space size and performance for different levels of morphological normalisation, and compared the results for both languages. Our experiments have shown that the improvements in SVM classifier performance is statistically significant; they are greater for small and medium number of features, especially for Croatian, whereas for large number of features the improvements are rather small and may be negligible in practice for both languages.
ISSN:0306-4573
1873-5371
DOI:10.1016/j.ipm.2006.12.007