Creating Prosodic Synchrony for a Robot Co-player in a Speech-controlled Game for Children

• Main Authors: Sadoughi, Najmeh, Pereira, André, Jain, Rishub, Leite, Iolanda, Lehman, Jill Fain
• Format: Conference Proceeding
• Language: English
• Subjects: Correlation; Feature extraction; Games; Human-centered computing > Collaborative and social computing > Empirical studies in collaborative and social computing; Human-centered computing > Human computer interaction (HCI) > Interactive systems and tools; Information systems > Information systems applications > Collaborative and social computing systems and tools; Robot kinematics; Robot sensing systems; Synchronization
• ISSN: 2167-2148
• DOI: 10.1145/2909824.3020244

Synchrony is an essential aspect of human-human interactions. In previous work, we have seen how synchrony manifests in low-level acoustic phenomena like fundamental frequency, loudness, and the duration of keywords during the play of child-child pairs in a fast-paced, cooperative, language-based game. The correlation between the increase in such low-level synchrony and increase in enjoyment of the game suggests that a similar dynamic between child and robot co-players might also improve the child's experience. We report an approach to creating on-line acoustic synchrony by using a dynamic Bayesian network learned from prior recordings of child-child play to select from a predefined space of robot speech in response to real-time measurement of the child's prosodic features. Data were collected from 40 new children, each playing the game with both a synchronizing and non-synchronizing version of the robot. Results show a significant order effect: although all children grew to enjoy the game more over time, those that began with the synchronous robot maintained their own synchrony to it and achieved higher engagement compared with those that did not.