Deployment strategies for crowdsourcing text creation

•Deployment strategies for text creation tasks were proposed.•Strategies were formalized by work structure, workforce organization and work style.•Recommendations on strategies to use when crowdsourcing text creation were provided. Automatically generating text of high quality in tasks such as trans...

Full description

Saved in:
Bibliographic Details
Published in:Information systems (Oxford) 2017-11, Vol.71, p.103-110
Main Authors: Borromeo, Ria Mae, Laurent, Thomas, Toyama, Motomichi, Alsayasneh, Maha, Amer-Yahia, Sihem, Leroy, Vincent
Format: Article
Language:English
Subjects:
Artificial intelligence
Automatic text analysis
Computer Science
Creativity
Crowdsourcing
Deployment strategies
Information systems
Social and Information Networks
Strategy
Text creation
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•Deployment strategies for text creation tasks were proposed.•Strategies were formalized by work structure, workforce organization and work style.•Recommendations on strategies to use when crowdsourcing text creation were provided. Automatically generating text of high quality in tasks such as translation, summarization, and narrative writing is difficult as these tasks require creativity, which only humans currently exhibit. However, crowdsourcing such tasks is still a challenge as they are tedious for humans and can require expert knowledge. We thus explore deployment strategies for crowdsourcing text creation tasks to improve the effectiveness of the crowdsourcing process. We consider effectiveness through the quality of the output text, the cost of deploying the task, and the latency in obtaining the output. We formalize a deployment strategy in crowdsourcing along three dimensions: work structure, workforce organization, and work style. Work structure can either be simultaneous or sequential, workforce organization independent or collaborative, and work style either by humans only or by using a combination of machine and human intelligence. We implement these strategies for translation, summarization, and narrative writing tasks by designing a semi-automatic tool that uses the Amazon Mechanical Turk API and experiment with them in different input settings such as text length, number of sources, and topic popularity. We report our findings regarding the effectiveness of each strategy and provide recommendations to guide requesters in selecting the best strategy when deploying text creation tasks.
ISSN:0306-4379
1873-6076
DOI:10.1016/j.is.2017.06.007