A novel machine learning analysis of eye-tracking data reveals suboptimal visual information extraction from facial stimuli in individuals with autism

• Published in: Neuropsychologia 2019-06, Vol.129, p.397-406
• Main Authors: Król, Magdalena Ewa, Król, Michał
• Format: Article
• Language: English
• Subjects: Autism spectrum disorder; Eye-tracking; Face emotion recognition; Face processing; Machine-learning
• ISSN: 0028-3932; 1873-3514
• DOI: 10.1016/j.neuropsychologia.2019.04.022

We propose a new method of quantifying the utility of visual information extracted from facial stimuli for emotion recognition. The stimuli are convolved with a Gaussian fixation distribution estimate, revealing more information in those facial regions the participant fixated on. Feeding this convolution to a machine-learning emotion recognition algorithm yields an error measure (between actual and predicted emotions) reflecting the quality of extracted information. We recorded the eye-movements of 21 participants with autism and 23 age-, sex- and IQ-matched typically developing participants performing three facial analysis tasks: free-viewing, emotion recognition, and brow-mouth width comparison. In the emotion recognition task, fixations of participants with autism were positioned on lower areas of the faces and were less focused on the eyes compared to the typically developing group. Additionally, the utility of information extracted by them in the emotion recognition task was lower. Thus, the emotion recognition deficit typical in autism can be at least partly traced to the earliest stage of face processing, i.e. to the extraction of visual information via eye-fixations. •When recognising emotions, people with autism looked at faces differently than controls.•Their face scanning patterns were more variable and less focused on the eyes.•Using machine learning we show that it hindered their ability to recognise emotions.•An emotion recognition algorithm was fed parts of face stimuli that participants looked at.•It was less effective when fed only those parts of faces that autistic people fixated on.