Improving Spectral Clustering with Deep Embedding and Cluster Estimation

• Main Authors: Duan, Liang, Aggarwal, Charu, Ma, Shuai, Sathe, Saket
• Format: Conference Proceeding
• Language: English
• Subjects: autoencoder; clustering; deep embedding; number of clusters; spectral clustering
• ISSN: 2374-8486
• DOI: 10.1109/ICDM.2019.00027

Spectral clustering is one of the most popular modern clustering algorithms. It is easy to implement, can be solved efficiently, and very often outperforms other traditional clustering algorithms such as k-means. However, pectral clustering would be insufficient when dealing with most datasets which have complex statistical properties and quires the user to specify the number of clusters (called k). To address these two problems, in this paper, we propose an approach to extending spectral clustering with deep embedding and estimation of the number of clusters. Specifically, we first generate the deep embedding via learning a deep autoencoder, which transforms the raw data into the lower dimensional representations that suitable for clustering. We then provide an effective method to estimate the number of clusters by learning a softmax autoencoder from the deep embedding. We finally extend spectral clustering with the learned embedding and the estimated number. An extensive experimental study on several image and text datasets illustrates the effectiveness and efficiency of our approach.