Postimpact similarity: a similarity measure for effective grouping of unlabelled text using spectral clustering

• Published in: Knowledge and information systems 2022-03, Vol.64 (3), p.723-742
• Main Authors: Roy, Arnab Kumar, Basu, Tanmay
• Format: Article
• Language: English
• Subjects: Clustering; Computer Science; Data Mining and Knowledge Discovery; Database Management; Documents; Information Storage and Retrieval; Information Systems and Communication Service; Information Systems Applications (incl.Internet); IT in Business; Regular Paper; Similarity; Similarity measures; Spectral methods
• ISSN: 0219-1377; 0219-3116
• DOI: 10.1007/s10115-022-01658-9

The task of text clustering is to partition a set of text documents into different meaningful groups such that the documents in a particular cluster are more similar to each other than the documents of other clusters according to a similarity or dissimilarity measure. Therefore, the role of similarity measure is crucial for producing good-quality clusters. The content similarity between two documents is generally used to form individual clusters, and it is measured by considering shared terms between the documents. However, the same may not be effective for a reasonably large and high-dimensional corpus. Therefore, a similarity measure is proposed here to improve the performance of text clustering using spectral method. The proposed similarity measure between two documents assigns a score based on their content similarity and their individual similarity with the shared neighbours over the corpus. The effectiveness of the proposed document similarity measure has been tested for clustering of different standard corpora using spectral clustering method. The empirical results using some well-known text collections have shown that the proposed method performs better than the state-of-the-art text clustering techniques in terms of normalized mutual information, f -measure and v -measure.