Similarity model and term association for document categorization

Both Euclidean distance- and cosine-based similarity models are widely used for measures of document similarity in information retrieval and document categorization. These two similarity models are based on the assumption that term vectors are orthogonal. But this assumption is not true. Term associ...

Full description

Saved in:
Bibliographic Details
Main Authors: Huaizhong Kou, Gardarin, G.
Format: Conference Proceeding
Language:English
Subjects:
Conferences
Euclidean distance
Expert systems
Humans
Information retrieval
Mathematical model
Nearest neighbor searches
Support vector machines
Text categorization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Both Euclidean distance- and cosine-based similarity models are widely used for measures of document similarity in information retrieval and document categorization. These two similarity models are based on the assumption that term vectors are orthogonal. But this assumption is not true. Term associations are ignored in such similarity models. In the document categorization context, we analyze the properties of term-document space, term-category space and category-document space. Then, without the assumption of term independence, we propose a new mathematical model to estimate the association between terms and define an /spl epsiv/-similarity model of documents. Here we make best use of existing category membership represented by the corpus as much as possible, and the objective is to improve categorization performance. Experiments have been done with a k-NN classifier over the Reuters-5178 corpus. The empirical results show that utilization of term association can improve the effectiveness of the categorization system and the /spl epsiv/-similarity model outperforms those without term association.
ISSN:1529-4188
2378-3915
DOI:10.1109/DEXA.2002.1045908