Exploring Retrospective Annotation in Long-Videos for Emotion Recognition

Emotion recognition systems are typically trained to classify a given psychophysiological state into emotion categories. Current platforms for emotion ground-truth collection show limitations for real-world scenarios of long-duration content (e.g. > > 10 minutes), namely: 1) Real-time annotat...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on affective computing 2024-07, Vol.15 (3), p.1514-1525
Main Authors: Bota, Patricia, Cesar, Pablo, Fred, Ana, da Silva, Hugo Placido
Format: Article
Language:English
Subjects:
Algorithms
Annotation
Annotations
Electroencephalography
Emotion recognition
Emotions
Motion pictures
physiological signals
Physiology
Real-time systems
retrospective
Segmentation
Streaming media
Taskload
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Emotion recognition systems are typically trained to classify a given psychophysiological state into emotion categories. Current platforms for emotion ground-truth collection show limitations for real-world scenarios of long-duration content (e.g. > > 10 minutes), namely: 1) Real-time annotation tools are distracting and become exhausting; 2) Perform retrospective annotation of the whole content in bulk (providing highly coarse annotations); or 3) Are used by external experts (depending on the number of annotators and their subjective experience). We explore a novel approach, the EmotiphAI Annotator, that allows undisturbed content visualisation and simplifies the annotation process by using segmentation algorithms that select brief clips for emotional annotation retrospectively. We compare three methods for content segmentation based on physiological data (Electrodermal Activity (EDA), emotion-based), scene (time-based), and random (control) selection. The EmotiphAI Annotator attained a B+ System Usability Scale score and low-average mental workload as per the NASA Task Load Index (40%). The reliability of the self-report was analysed by the inter-rater agreement (STD <
ISSN:1949-3045
1949-3045
DOI:10.1109/TAFFC.2024.3359706