Quality-Assured Synchronized Task Assignment in Crowdsourcing

With the rapid development of crowdsourcing platforms that aggregate the intelligence of Internet workers, crowdsourcing has been widely utilized to address problems that require human cognitive abilities. Considering great dynamics of worker arrival and departure, it is of vital importance to desig...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2018-06
Main Authors: Tu, Jiayang, Cheng, Peng, Chen, Lei
Format: Article
Language:English
Subjects:
Algorithms
Approximation
Benchmarks
Crowdsourcing
Human performance
Intelligence
Parameter estimation
Quality assurance
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:With the rapid development of crowdsourcing platforms that aggregate the intelligence of Internet workers, crowdsourcing has been widely utilized to address problems that require human cognitive abilities. Considering great dynamics of worker arrival and departure, it is of vital importance to design a task assignment scheme to adaptively select the most beneficial tasks for the available workers. In this paper, in order to make the most efficient utilization of the worker labor and balance the accuracy of answers and the overall latency, we a) develop a parameter estimation model that assists in estimating worker expertise, question easiness and answer confidence; b) propose a \textit{quality-assured synchronized task assignment scheme} that executes in batches and maximizes the number of potentially completed questions (MCQ) within each batch. We prove that MCQ problem is NP-hard and present two greedy approximation solutions to address the problem. The effectiveness and efficiency of the approximation solutions are further evaluated through extensive experiments on synthetic and real datasets. The experimental results show that the accuracy and the overall latency of the MCQ approaches outperform the existing online task assignment algorithms in the synchronized task assignment scenario.
ISSN:2331-8422