Deep Embedded Clustering with ResNets

• Main Authors: Wickramasinghe, Chathurika, Marino, Daniel, Manic, Milos
• Format: Conference Proceeding
• Language: English
• Subjects: Activity recognition; Autoencoders; Benchmark testing; Clustering; Data mining; Deep Embedded Clustering; Deep learning; Deep Neural Networks; Degradation; Feature extraction; Neural networks; ResNet; Unsupervised Learning
• ISSN: 2158-2254
• DOI: 10.1109/HSI52170.2021.9538747

Clustering is an AI technique that has been successfully applied to the abundance of unlabelled real-world data for revealing hidden patterns and knowledge extraction. Deep Embedded Clustering (DEC) is a deep Autoencoder (AE) based model that learns feature representations and cluster assignments simultaneously. DEC learns the mapping from input data to a low-dimensional embedded space through joint optimization of feature transformation and clustering. Our previous work demonstrates how adding residual connections to deep AEs (RAEs) reduces the performance degradation of learned features when performing downstream classification on learned features. Further, it evidenced that RAE has improved unsupervised feature learning capability compared to AE. In this paper, we are evaluating the effect of residual connections in the context of Deep Embedded Clustering (DEC), which we refer to as RDEC. RDEC was compared against regular DEC. We considered various numbers of hidden layers and several bench-mark datasets: MNIST, Fashion MNIST, Reuters, and Human activity recognition. When increasing the depth of the neural network gradually, the presented RDEC showed up to 56% of less performance degradation compared to DEC. Further, the distribution of clustering accuracies showed that the presented RDEC outperforms DEC when comparing the accuracy variance and mean accuracy.