Web document clustering based on a new niching Memetic Algorithm, Term-Document Matrix and Bayesian Information Criterion

This paper introduces a new description-centric algorithm for web document clustering based on Memetic Algorithms with Niching Methods, Term-Document Matrix and Bayesian Information Criterion. The algorithm defines the number of clusters automatically. The Memetic Algorithm provides a combined globa...

Full description

Saved in:
Bibliographic Details
Main Authors: Cobos, C, Montealegre, C, Mejia, M, Mendoza, M, Leon, E
Format: Conference Proceeding
Language:English
Subjects:
Algorithm design and analysis
Classification algorithms
Clustering algorithms
Complexity theory
Heuristic algorithms
Memetics
Partitioning algorithms
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper introduces a new description-centric algorithm for web document clustering based on Memetic Algorithms with Niching Methods, Term-Document Matrix and Bayesian Information Criterion. The algorithm defines the number of clusters automatically. The Memetic Algorithm provides a combined global and local strategy for a search in the solution space and the Niching methods to promote diversity in the population and prevent the population from converging too quickly (based on restricted competition replacement and restrictive mating). The Memetic Algorithm uses the K-means algorithm to find the optimum value in a local search space. Bayesian Information Criterion is used as a fitness function, while FP-Growth is used to reduce the high dimensionality in the vocabulary. This resulting algorithm, called WDC-NMA, was tested with data sets based on Reuters-21578 and DMOZ, obtaining promising results (better precision results than a Singular Value Decomposition algorithm). Also, it was also then initially evaluated by a group of users.
ISSN:1089-778X
1941-0026
DOI:10.1109/CEC.2010.5586016