Automatic discovery of locally frequent itemsets in the presence of highly frequent itemsets

• Published in: Intelligent data analysis 2005-01, Vol.9 (1), p.83-104
• Main Authors: Bodon, Ferenc, Kouris, Ioannis N., Makris, Christos H., Tsakalidis, Athanasios K.
• Format: Article
• Language: English
• ISSN: 1088-467X; 1571-4128
• DOI: 10.3233/IDA-2005-9106

Many alternatives have been proposed for the mining of association rules involving rare but 'interesting' itemsets in a dataset where there also exist highly frequent itemsets. Nevertheless, all the approaches thus far suggested that we knew which those interesting itemsets are, as well as which is the right support value for them. None of the approaches proposed a way of automatically discovering such items. In this work we introduce the notion of locally frequent itemsets and support their existence as the biggest and most frequently appearing category of rare but interesting itemsets especially at commercial applications, based on the opinion of field experts. Subsequently we propose two algorithms for finding and handling these itemsets. The main idea is to divide the database into partitions according to the problem needs and besides searching for itemsets which are frequent in the whole database to search also for itemsets which are frequent if considered within these partitions. Our approach proves very effective and also very efficient as compared to the traditional algorithms both in synthetic and real data.