An Integrated Approach for Making Inference on the Number of Clusters in a Mixture Model

• Published in: Entropy (Basel, Switzerland) Switzerland), 2019-11, Vol.21 (11), p.1063
• Main Authors: Saraiva, Erlandson Ferreira, Suzuki, Adriano Kamimura, Milan, Luis Aparecido, Pereira, Carlos Alberto de Bragança
• Format: Article
• Language: English
• Subjects: Algorithms; Clustering; Datasets; em algorithm; Estimates; Integrated approach; Iterative methods; Maximization; mixture model; model-based clustering; Optimization; Parameter estimation; Probabilistic models; Probability; Variables
• ISSN: 1099-4300; 1099-4300
• DOI: 10.3390/e21111063

This paper presents an integrated approach for the estimation of the parameters of a mixture model in the context of data clustering. The method is designed to estimate the unknown number of clusters from observed data. For this, we marginalize out the weights for getting allocation probabilities that depend on the number of clusters but not on the number of components of the mixture model. As an alternative to the stochastic expectation maximization (SEM) algorithm, we propose the integrated stochastic expectation maximization (ISEM) algorithm, which in contrast to SEM, does not need the specification, a priori, of the number of components of the mixture. Using this algorithm, one estimates the parameters associated with the clusters, with at least two observations, via local maximization of the likelihood function. In addition, at each iteration of the algorithm, there exists a positive probability of a new cluster being created by a single observation. Using simulated datasets, we compare the performance of the ISEM algorithm against both SEM and reversible jump (RJ) algorithms. The obtained results show that ISEM outperforms SEM and RJ algorithms. We also provide the performance of the three algorithms in two real datasets.