Crowdsourcing aggregation with deep Bayesian learning

• Published in: Science China. Information sciences 2021-03, Vol.64 (3), p.130104, Article 130104
• Main Authors: Li, Shao-Yuan, Huang, Sheng-Jun, Chen, Songcan
• Format: Article
• Language: English
• Subjects: Algorithms; Annotations; Artificial neural networks; Back propagation; Bayesian analysis; Classifiers; Computer Science; Crowdsourcing; Inference; Information Systems and Communication Service; Labels; Machine learning; Message passing; Neural networks; Optimization techniques; Probabilistic models; Probability theory; Regression analysis; Research Paper; Workers
• ISSN: 1674-733X; 1869-1919
• DOI: 10.1007/s11432-020-3118-7

In this study, we consider a crowdsourcing classification problem in which labeling information from crowds is aggregated to infer latent true labels. We propose a fully Bayesian deep generative crowdsourcing model (BayesDGC), which combines the strength of deep neural networks (DNNs) on automatic representation learning and the interpretable probabilistic structure encoding of probabilistic graphical models. The model comprises a DNN classifier as a prior for the true labels and a probabilistic model for the annotation generation process. The DNN classifier and annotation generation process share the latent true label variables. To address the inference challenge, we developed a natural-gradient stochastic variational inference, which combines variational message passing for conjugate parameters and stochastic gradient descent for DNN and learns the distribution of latent true labels and workers’ confusion matrix via end-to-end training. We illustrated the effectiveness of the proposed model using empirical results on 22 real-world datasets.