Text clustering approach based on maximal frequent term sets

• Main Authors: Chong Su, Qingcai Chen, Xiaolong Wang, Xianjun Meng
• Format: Conference Proceeding
• Language: English
• Subjects: Clustering algorithms; Cybernetics; Data mining; Frequent Term Sets; Large-scale systems; Maximal Frequent Term Sets; Motion pictures; Performance gain; Scalability; Space technology; Text Clustering; Text mining; USA Councils
• ISSN: 1062-922X; 2577-1655
• DOI: 10.1109/ICSMC.2009.5346313

Classical text clustering algorithms are usually based on vector space model or its variants. Because of the high computing complexity and the difficulty of controlling clustering results, this kind of approaches are hard to be applied for the purpose of the large scale text clustering. Clustering algorithms based on frequent term sets make use of relationship among documents and their shared frequent term sets to achieve high accuracy and effectiveness in clustering. But since the number of frequent terms is usually too large to reach the efficiency requirement for large collection texts clustering, this paper proposes a novel text clustering approach based on maximal frequent term sets (MFTSC). This approach firstly mines maximal frequent term sets from text set and then clusters texts by following steps: at first, the maximal frequent term sets are clustered based on the criterion of k-mismatch; then texts are clustered according to term sets clustering results; finally, we categorize the left texts uncovered in previous step into produced text clusters Be compared with existing approaches, our experimental results show an average gain of 10% on F-Measure score with better performance on scalability and efficiency.