Harnessing Multimodal Data and Deep Learning for Comprehensive Gait Analysis in Pediatric Cerebral Palsy

• Published in: IEEE transactions on consumer electronics 2024-08, Vol.70 (3), p.5401-5410
• Main Authors: Yang, Jing, Li, Liangyu, Por, Lip Yee, Bourouis, Sami, Dhahbi, Sami, Ayub Khan, Abdullah
• Format: Article
• Language: English
• Subjects: Abnormalities; Accuracy; Artificial neural networks; Cerebral palsy; Consumer electronics; Convolutional neural networks; Deep learning; deep neural architecture; Diagnosis; Gain measurement; Gait; gait analysis; Inertial platforms; Inertial sensing devices; Machine learning; Magnetic resonance imaging; Motion capture; Motors; multimodal data; Muscles; pediatric neurological disorder; Pediatrics; wearable sensors
• ISSN: 0098-3063; 1558-4127
• DOI: 10.1109/TCE.2024.3482689

Cerebral palsy (CP) is a leading cause of motor dysfunction in children, significantly impacting gait and mobility. Accurate and early diagnosis of gait abnormalities in pediatric CP patients is crucial for effective intervention and management. However, making an early-stage CP diagnosis based only on a single vision modality such as an MRI has many difficulties. Because of the baby's obstinate movements, the possibility of early recovery, the lack of a single vision modality, and the noisy or absent brain magnetic resonance imaging (MRI) slices, the task is getting harder and harder. This study employed a robust framework that leverages data from multiple sensor modalities, including wearable inertial measurement units (IMUs), pressure-sensitive mats, and motion capture systems integrated with MRI to generate multimodal data. This multimodal data was then processed using convolutional neural networks (CNNs) and long short-term memory (LSTM) networks to capture both spatial and temporal dynamics of gait patterns. In the experimentation, we achieved remarkable results with an accuracy of 95.33%, an AUC of 96.2%, an F1 score of 95.28%, and a misclassification rate of 0.0467. Also, the comparative analysis with state-of-the-art demonstrates that the proposed approach significantly outperforms traditional methods in identifying subtle gait abnormalities, providing a more detailed and accurate assessment of gait deviations in pediatric cerebral palsy patients.