Introducing the Cosine Clustering Index (CCI): A Balanced Approach to Evaluating Deep Clustering

• Published in: SN computer science 2024-07, Vol.5 (6), p.687, Article 687
• Main Authors: Jahanian, Mojtaba, Karimi, Abbas, Osati Eraghi, Nafiseh, Zarafshan, Faraneh
• Format: Article
• Language: English
• Subjects: Algorithms; Big Data; Clustering; Clusters; Computer Imaging; Computer Science; Computer Systems Organization and Communication Networks; Data Structures and Information Theory; Datasets; Deep learning; Efficiency; Euclidean geometry; Information Systems and Communication Service; Machine learning; Original Research; Pattern Recognition and Graphics; Software Engineering/Programming and Operating Systems; Vision
• ISSN: 2661-8907; 2662-995X; 2661-8907
• DOI: 10.1007/s42979-024-02970-7

Amidst the surge of Big Data, deep clustering emerges as a pivotal technique in machine learning, necessitating robust and interpretable evaluation metrics that align with its complexities. Traditional metrics, largely dependent on Euclidean distance, falter in capturing the essence of deep clustering in high-dimensional data. This paper proposes the Cosine Clustering Index (CCI), an innovative metric that leverages cosine distance to accurately assess clustering outcomes in deep learning contexts. Distinguished by its ability to evaluate intra-cluster cohesion and inter-cluster separation, CCI addresses the intricacies of deep clustering more effectively than existing metrics. Validation on benchmark datasets like IRIS, MNIST, and CIFAR10 showcases CCI’s enhanced interpretability, scalability, and its superior adaptability to the depth and dimensionality of data. The introduction of CCI marks a significant stride towards advancing clustering evaluation, promising a more profound understanding and analysis of deep clustering mechanisms.