Medium-term speaker states—A review on intoxication, sleepiness and the first challenge

• Published in: Computer speech & language 2014-03, Vol.28 (2), p.346-374
• Main Authors: Schuller, Björn, Steidl, Stefan, Batliner, Anton, Schiel, Florian, Krajewski, Jarek, Weninger, Felix, Eyben, Florian
• Format: Article
• Language: English
• Subjects: Challenge; Computational paralinguistics; Intoxication; Sleepiness; Survey
• ISSN: 0885-2308; 1095-8363
• DOI: 10.1016/j.csl.2012.12.002

► We review the state of the art in automatic recognition of intoxication and sleepiness. ► We present the framework and results of the INTERSPEECH 2011 Speaker State Challenge. ► We review the contributions of the Challenge participants and analyse their results. ► We highlight the opportunities by result fusion among systems and along the time axis. In the emerging field of computational paralinguistics, most research efforts are devoted to either short-term speaker states such as emotions, or long-term traits such as personality, gender, or age. To bridge this gap on the time axis, and hence broaden the scope of the field, the INTERSPEECH 2011 Speaker State Challenge addressed the algorithmic analysis of medium-term speaker states: alcohol intoxication and sleepiness, both of which are highly relevant in high risk environments. Preserving the paradigms of the two previous INTERSPEECH Challenges, researchers were invited to participate in a large-scale evaluation providing unified testing conditions. This article reviews previous efforts to automatically recognise intoxication and sleepiness from speech signals, and gives an overview on the Challenge conditions and data sets, the methods used by the participants, and their results. By fusing participants’ systems, we show that binary classification of alcoholisation and sleepiness from short-term observations, i.e., single utterances, can both reach over 72% accuracy on unseen test data; furthermore, we demonstrate that these medium-term states can be recognised more robustly by fusing short-term classifiers along the time axis, reaching up to 91% accuracy for intoxication and 75% for sleepiness.