Frequent Itemset Mining in Big Data With Effective Single Scan Algorithms

• Published in: IEEE access 2018, Vol.6, p.68013-68026
• Main Authors: Djenouri, Youcef, Djenouri, Djamel, Lin, Jerry Chun-Wei, Belhadi, Asma
• Format: Article
• Language: English
• Subjects: Algorithms; Apriori; Big Data; Clustering algorithms; Computer science; Data mining; frequent itemset mining; heuristic; Itemsets; parallel computing; Runtime; support computing
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2018.2880275

This paper considers frequent itemsets mining in transactional databases. It introduces a new accurate single scan approach for frequent itemset mining (SSFIM), a heuristic as an alternative approach (EA-SSFIM), as well as a parallel implementation on Hadoop clusters (MR-SSFIM). EA-SSFIM and MR-SSFIM target sparse and big databases, respectively. The proposed approach (in all its variants) requires only one scan to extract the candidate itemsets, and it has the advantage to generate a fixed number of candidate itemsets independently from the value of the minimum support. This accelerates the scan process compared with existing approaches while dealing with sparse and big databases. Numerical results show that SSFIM outperforms the state-of-the-art FIM approaches while dealing with medium and large databases. Moreover, EA-SSFIM provides similar performance as SSFIM while considerably reducing the runtime for large databases. The results also reveal the superiority of MR-SSFIM compared with the existing HPC-based solutions for FIM using sparse and big databases.