Emotion classification via utterance-level dynamics: A pattern-based approach to characterizing affective expressions

• Main Authors: Yelin Kim, Provost, Emily Mower
• Format: Conference Proceeding
• Language: English
• Subjects: Accuracy; dynamic pattern; dynamic time warping; emotion classification; emotion dynamics; Emotion recognition; emotion structure; Hidden Markov models; Mathematical model; multimodal; Speech; Speech recognition; Trajectory
• ISSN: 1520-6149; 2379-190X
• DOI: 10.1109/ICASSP.2013.6638344

Human emotion changes continuously and sequentially. This results in dynamics intrinsic to affective communication. One of the goals of automatic emotion recognition research is to computationally represent and analyze these dynamic patterns. In this work, we focus on the global utterance-level dynamics. We are motivated by the hypothesis that global dynamics have emotion-specific variations that can be used to differentiate between emotion classes. Consequently, classification systems that focus on these patterns will be able to make accurate emotional assessments. We quantitatively represent emotion flow within an utterance by estimating short-time affective characteristics. We compare time-series estimates of these characteristics using Dynamic Time Warping, a time-series similarity measure. We demonstrate that this similarity can effectively recognize the affective label of the utterance. The similarity-based pattern modeling outperforms both a feature-based baseline and static modeling. It also provides insight into typical high-level patterns of emotion. We visualize these dynamic patterns and the similarities between the patterns to gain insight into the nature of emotion expression.