Context aware stopwords for Sinhala Text classification

When working with Text Classification (TC), often the term "stopword" can be heard. Words in a document that are frequently occurring, but meaningless in terms of Information Retrieval (IR) are called Stopwords. There are various stopword lists available for many languages. According to th...

Full description

Saved in:
Bibliographic Details
Main Authors: Gunasekara, S.V.S, Haddela, Prasanna S.
Format: Conference Proceeding
Language:English
Subjects:
average accuracy
average Fmeasure
Information retrieval
Information technology
Sinhala
stopwords
Text categorization
threshold value
Time-frequency analysis
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:When working with Text Classification (TC), often the term "stopword" can be heard. Words in a document that are frequently occurring, but meaningless in terms of Information Retrieval (IR) are called Stopwords. There are various stopword lists available for many languages. According to the best of knowledge, no any generic stopword list has been built for the Sinhala language. This paper demonstrates how to generate a domain-specific stopword list from a given data set of Sinhala Newspapers. Hence, the seven stopword identification methods previously applied to other languages are presented to remove stopwords. Then, a new algorithm for building a domain-specific stopword list is proposed. For this method, it is assumed that average F-measure and average accuracy for the set of different stopword lists are measured by the performance of two classifiers. Based on the given comparative study, the most effective method to classify stopwords in Sinhala corpus can be identified.
ISSN:2279-3895
DOI:10.1109/NITC.2018.8550073