Embodied conversational agents for multimodal automated social skills training in people with autism spectrum disorders

• Published in: PloS one 2017-08, Vol.12 (8), p.e0182151-e0182151
• Main Authors: Tanaka, Hiroki, Negoro, Hideki, Iwasaka, Hidemi, Nakamura, Satoshi
• Format: Article
• Language: English
• Subjects: Acoustics; Adult; Adults; Autism; Autism Spectrum Disorder - psychology; Autism Spectrum Disorder - rehabilitation; Autistic children; Automation; Behavior; Behavior Therapy - methods; Biology and Life Sciences; CAI; Care and treatment; Children; Cognition & reasoning; Communication; Communication skills; Communications systems; Computer assisted instruction; Computer simulation; Computers; Conversation; Disorders; Feedback; Female; Frequency; Graduate students; Health aspects; Human performance; Humans; Information science; International conferences; Interpersonal Relations; Male; Medicine and Health Sciences; People and Places; Pitch (inclination); Public speaking; Rehabilitation; Scores; Sensory integration; Skill development; Social aspects; Social interaction; Social Sciences; Social Skills; Socialization; Speaking; Teachers; Teaching; Therapists; Training; Visual perception; Work skills; Yaw; Young Adult; Young adults
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0182151

Social skills training, performed by human trainers, is a well-established method for obtaining appropriate skills in social interaction. Previous work automated the process of social skills training by developing a dialogue system that teaches social communication skills through interaction with a computer avatar. Even though previous work that simulated social skills training only considered acoustic and linguistic information, human social skills trainers take into account visual and other non-verbal features. In this paper, we create and evaluate a social skills training system that closes this gap by considering the audiovisual features of the smiling ratio and the head pose (yaw and pitch). In addition, the previous system was only tested with graduate students; in this paper, we applied our system to children or young adults with autism spectrum disorders. For our experimental evaluation, we recruited 18 members from the general population and 10 people with autism spectrum disorders and gave them our proposed multimodal system to use. An experienced human social skills trainer rated the social skills of the users. We evaluated the system's effectiveness by comparing pre- and post-training scores and identified significant improvement in their social skills using our proposed multimodal system. Computer-based social skills training is useful for people who experience social difficulties. Such a system can be used by teachers, therapists, and social skills trainers for rehabilitation and the supplemental use of human-based training anywhere and anytime.