Human Expert Labeling Process (HELP): Towards a Reliable Higher-Order User State Labeling Process and Tool to Assess Student Engagement

In a series of longitudinal research studies, researchers at Intel Corporation in Turkey have been working towards an adaptive learning system automatically detecting student engagement as a higher-order user state in real-time. The labeled data necessary for supervised learning can be obtained thro...

Full description

Saved in:
Bibliographic Details
Published in:Educational technology 2017-01, Vol.57 (1), p.53-59
Main Authors: Aslan, Sinem, Mete, Sinem Emine, Okur, Eda, Oktay, Ece, Alyuz, Nese, Genc, Utku Ergin, Stanhill, David, Esme, Asli Arslan
Format: Article
Language:English
Subjects:
Classroom observations
Corporations
Data collection
Educational research
Emotional states
Expertise
Foreign Countries
Labeling (of Persons)
Learner Engagement
Longitudinal Studies
Machine learning
Observational research
Regular Issue Articles
Research studies
Research tools
Researchers
Thinking Skills
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In a series of longitudinal research studies, researchers at Intel Corporation in Turkey have been working towards an adaptive learning system automatically detecting student engagement as a higher-order user state in real-time. The labeled data necessary for supervised learning can be obtained through labeling conducted by human exp multiple labelers to label collected data and obtaining agreement among different labelers on the same samples of data, it is critical to train all to use the engagement model accurately. Addressing these challenges, the researchers developed a rigorous human expert labeling process (HELP) specific to the educational context, with multi-faceted labels and multiple expert labelers. HELP has three distinct stages: (1) Pre-Labeling, including planning, labeler recruitment, training, and evaluation steps; (2) Labeling, involving actual labeling conducted by multiple labelers, and related steps for formative assessment of their performance; and (3) Post-Label ing, generating final labels and agreement measures through processing multiple decisions. In this article, the researchers outline proposed methods in HELP and describe the developed labeling tool.
ISSN:0013-1962