Vision-assisted recognition of stereotype behaviors for early diagnosis of Autism Spectrum Disorders

• Published in: Neurocomputing (Amsterdam) 2021-07, Vol.446, p.145-155
• Main Authors: Negin, Farhood, Ozyer, Baris, Agahian, Saeid, Kacdioglu, Sibel, Ozyer, Gulsah Tumuklu
• Format: Article
• Language: English
• Subjects: Action recognition; Autism Spectrum Disorder; Bag-of-visual-words; Convolutional neural networks; Patient monitoring
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2021.03.004

•A video dataset of Autism Spectrum Disorders (ASD) is collected.•Possibility of diagnosis of ASD is analyzed by developed action recognition methods.•The evaluations emphasize the effectiveness of appearance-based local descriptors.•According to the clinicians, our method can help them in the early diagnosis of ASD. Medical diagnosis supported by computer-assisted technologies is getting more popularity and acceptance among medical society. In this paper, we propose a non-intrusive vision-assisted method based on human action recognition to facilitate the diagnosis of Autism Spectrum Disorder (ASD). We collected a novel and comprehensive video dataset f the most distinctive Stereotype actions of this disorder with the assistance of professional clinicians. Several frameworks as a function of different input modalities were developed and used to produce extensive baseline results. Various local descriptors, which are commonly used within the Bag-of-Visual-Words approach, were tested with Multi-layer Perceptron (MLP), Gaussian Naive Bayes (GNB), and Support Vector Machines (SVM) classifiers for recognizing ASD associated behaviors. Additionally, we developed a framework that first receives articulated pose-based skeleton sequences as input and follows an LSTM network to learn the temporal evolution of the poses. Finally, obtained results were compared with two fine-tuned deep neural networks: ConvLSTM and 3DCNN. The results revealed that the Histogram of Optical Flow (HOF) descriptor achieves the best results when used with MLP classifier. The promising baseline results also confirmed that an action-recognition-based system can be potentially used to assist clinicians to provide a reliable, accurate, and timely diagnosis of ASD disorder.