Fostering emotion recognition in children with autism spectrum disorder

• Published in: Multimodal technologies and interaction 2021-10, Vol.5 (10), p.57-75
• Main Authors: Silva, Vinicius Corrêa Alves, Soares, Filomena, Esteves, João Sena, Santos, Cristina, Pereira, Ana Paula da Silva
• Format: Article
• Language: English
• Subjects: Autism; Autism Spectrum Disorder; Autistic children; Behavior; Educação de qualidade; Eletrónica e Informática; Emotion recognition; Emotions; Emotions Recognition; Engenharia e Tecnologia; Engenharia Eletrotécnica; Facial Expressions; Feature extraction; Happiness; Human Com Support Vector Machine; Human Computer Interaction; Robotics; Robots; Science & Technology; Social factors; Support Vector Machine; Support vector machines; Verbal communication; Zeno R50 Robot
• ISSN: 2414-4088; 2414-4088
• DOI: 10.3390/mti5100057

Facial expressions are of utmost importance in social interactions, allowing communicative prompts for a speaking turn and feedback. Nevertheless, not all have the ability to express themselves socially and emotionally in verbal and non-verbal communication. In particular, individuals with Autism Spectrum Disorder (ASD) are characterized by impairments in social communication, repetitive patterns of behaviour, and restricted activities or interests. In the literature, the use of robotic tools is reported to promote social interaction with children with ASD. The main goal of this work is to develop a system capable of automatic detecting emotions through facial expressions and interfacing them with a robotic platform (Zeno R50 Robokind® robotic platform, named ZECA) in order to allow social interaction with children with ASD. ZECA was used as a mediator in social communication activities. The experimental setup and methodology for a real-time facial expression (happiness, sadness, anger, surprise, fear, and neutral) recognition system was based on the Intel® RealSense™ 3D sensor and on facial features extraction and multiclass Support Vector Machine classifier. The results obtained allowed to infer that the proposed system is adequate in support sessions with children with ASD, giving a strong indication that it may be used in fostering emotion recognition and imitation skills. Centro Algoritmi & CIEd. CT—Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020. Vinicius Silva thanks FCT for the PhD scholarship SFRH/BD/SFRH/BD/133314/2017.