RGB-D feature extraction method for hand gesture recognition based on a new fast and accurate multi-channel cartesian Jacobi moment invariants

• Published in: Multimedia tools and applications 2022-04, Vol.81 (9), p.12725-12757
• Main Authors: El Ouariachi, Ilham, Benouini, Rachid, Zenkouar, Khalid, Zarghili, Arsalane, El Fadili, Hakim
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Cartesian coordinates; Complexity; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Deep learning; Experiments; Feature extraction; Gesture recognition; Human-computer interaction; Image reconstruction; Invariants; Machine learning; Methods; Multimedia; Multimedia Information Systems; Numerical stability; Object recognition; Pattern recognition; Sign language; Special Purpose and Application-Based Systems
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-022-12161-2

Due to the diversity of hand gestures uses in human computer interaction and the complexity involved by gestures, many features have been proposed, however each feature has its own drawbacks. Therefore, in this work, we propose a new set of Red, Green, Blue and Depth (RGB-D) feature extraction method based on Image Moment Invariants, named Fast and Accurate Multi-channel Cartesian Jacobi Moment Invariants for Depth (FA-MCJMI D ), RGB (FA-MCJMI R G B ) and RGB-D (FA-MCJMI R G B D ) images. We first introduce the fundamental concepts and properties to present the Multi-channel Cartesian Jacobi Moments (MCJMs). Then, we express the MCJMIs using geometric moment invariants under Rotation, Scaling and Translation (RST) transforms. Moreover, we provide the theoretical approach to enhance their numerical accuracy and improve their computational speed. Then we explore the application of our new moment invariants in hand gesture representation and recognition. Accordingly, several experiments are conducted to validate this new set of FA-MCJMI in comparison with some deep learning approches and other existing methods, on several popular hand gesture datasets, with regard to image reconstruction, invariability, numerical stability, computational complexity and recognition. The experiments demonstrate the superiority of the new FA-MCJMI set over the methods commonly used in the literature under geometric distortions, illumination variations and image occlusions.