A thorough experimental study of datasets for frequent itemsets

The discovery of frequent patterns is a famous problem in data mining. While plenty of algorithms have been proposed during the last decade, only a few contributions have tried to understand the influence of datasets on the algorithms behavior. Being able to explain why certain algorithms are likely...

Full description

Saved in:
Bibliographic Details
Main Authors: Flouvat, F., De March, F., Petit, J.M.
Format: Conference Proceeding
Language:English
Subjects:
Adaptive algorithm
Algorithm design and analysis
Association rules
Classification algorithms
Conferences
Data mining
Data structures
Itemsets
Statistical distributions
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The discovery of frequent patterns is a famous problem in data mining. While plenty of algorithms have been proposed during the last decade, only a few contributions have tried to understand the influence of datasets on the algorithms behavior. Being able to explain why certain algorithms are likely to perform very well or very poorly on some datasets is still an open question. In this setting, we describe a thorough experimental study of datasets with respect to frequent item sets. We study the distribution of frequent item sets with respect to item sets size together with the distribution of three concise representations: frequent closed, frequent free and frequent essential item sets. For each of them, we also study the distribution of their positive and negative borders whenever possible. From this analysis, we exhibit a new characterization of datasets and some invariants allowing to better predict the behavior of well known algorithms. The main perspective of this work is to devise adaptive algorithms with respect to dataset characteristics.
ISSN:1550-4786
2374-8486
DOI:10.1109/ICDM.2005.15