A signer-independent Arabic Sign Language recognition system using face detection, geometric features, and a Hidden Markov Model

• Published in: Computers & electrical engineering 2012-03, Vol.38 (2), p.422-433
• Main Authors: Mohandes, M., Deriche, M., Johar, U., Ilyas, S.
• Format: Article
• Language: English
• Subjects: Algorithms; Face recognition; Gaussian; Mathematical models; Movements; Recognition; Tracking
• ISSN: 0045-7906; 1879-0755
• DOI: 10.1016/j.compeleceng.2011.10.013

[Display omitted] ► Many sign language recognition systems exist but very few deal with Arabic language. ► The face region is first detected using a Gaussian skin model. ► An optimal set of geometric features is then extracted from the hands regions. ► Classification is performed using HMM with optimized number of states and modes. ► Recognition accuracy of above 95% was achieved with 300 Arabic signs. In this paper, we propose an image-based system for Arabic Sign Language (ArSL) recognition. The algorithm starts by detecting the face of the signer using a Gaussian skin color model. The centroid of the detected face is then used as a reference point for tracking the hands’ movements. The hands regions are segmented using a region growing algorithm assuming the signer wears a yellow and an orange colored gloves. From the segmented hands regions, an optimal set of features is extracted. To represent the time varying feature patterns, a Hidden Markov Model (HMM) is then used. Before using HMM in testing, the number of states and the number of Gaussian mixtures are optimized. The proposed system was implemented for both signer dependent and signer independent conditions. The experimental results show that an accuracy of more than 95% can be achieved with a large database of 300 signs. The results outperform previous work on ArSL mainly restricted to small vocabulary size.