Self-talk Discrimination in Human–Robot Interaction Situations for Supporting Social Awareness

• Published in: International journal of social robotics 2013-04, Vol.5 (2), p.277-289
• Main Authors: Le Maitre, Jade, Chetouani, Mohamed
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Cognitive science; Cognitive tasks; Computer Science; Computer Vision and Pattern Recognition; Control; Decision trees; Discrimination; Engineering; Feature recognition; Linguistics; Mechatronics; Neuroscience; Older people; Psychology; Robotics; Robots; Signal and Image Processing; Stimulation; Support vector machines
• ISSN: 1875-4791; 1875-4805
• DOI: 10.1007/s12369-013-0179-x

Being aware of the presence, activities and is fundamental for Human–Robot Interaction and assistive applications. In this paper, we describe (1) designing triadic situations for cognitive stimulation for elderly users; (2) characterizing social signals that describe social context: system directed speech (SDS) and self-talk (ST); and (3) estimating an interaction efficiency measure that reveals the quality of interaction. The proposed triadic situation is formed by a user, a computer providing cognitive exercises and a robot that provides encouragement and help using verbal and non-verbal signals. The methodology followed to design this situation is presented. Wizard-of-Oz experiments have been performed and analyzed through eye-contact behaviors and dialog acts (SDS and ST). We show that users employ two interaction styles characterized by different prosody features. Automatic recognition systems of these dialog acts is proposed using k -NN, decision tree and SVM classifiers trained with pitch, energy and rhythmic-based features. The best recognition system achieves an accuracy of 71 %, showing that interaction styles can be discriminated on the basis of prosodic features. An Interaction Efficiency (IE) metric is proposed to characterize interaction styles. This metric exploits on-view/off-view discrimination, semantic analysis and ST/SDS discrimination. Experiments on collected data prove the effectiveness of the IE measure in evaluating the individual’s quality of interaction of elderly patients during the cognitive stimulation task.