Histogram-Based Dimensionality Reduction of Term Vector Space

One of the most vital problems of free-text document processing is the curse of dimensionality. The paper presents a dimensionality reduction algorithm based on informed feature selection. Terms describing the document are based on histogram-like statistics which can be computed as well as increment...

Full description

Saved in:
Bibliographic Details
Main Authors: Ciesielski, Krzysztof, Klopotek, Mieczyslaw A., Wierzchon, Slawomir T.
Format: Conference Proceeding
Language:English
Subjects:
Computer science
Discrete transforms
Frequency
Image analysis
Karhunen-Loeve transforms
Matrix decomposition
Multidimensional systems
Principal component analysis
Sparse matrices
Statistics
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:One of the most vital problems of free-text document processing is the curse of dimensionality. The paper presents a dimensionality reduction algorithm based on informed feature selection. Terms describing the document are based on histogram-like statistics which can be computed as well as incrementally updated at low complexity. The document representation can adapt to changing document collection characteristics. Along with the fundamental concepts we present an empirical verification of the approach.
DOI:10.1109/CISIM.2007.35