Human action recognition using shape and CLG-motion flow from multi-view image sequences

• Published in: Pattern recognition 2008-07, Vol.41 (7), p.2237-2252
• Main Authors: Ahmad, Mohiuddin, Lee, Seong-Whan
• Format: Article
• Language: English
• Subjects: Action matrix; Action recognition; Applied sciences; Combined local–global (CLG) optic flow; Exact sciences and technology; Image processing; Information, signal and communications theory; Invariant Zernike moments; Multi-view image sequence; Multidimensional hidden Markov model (MDHMM); Signal processing; Telecommunications and information theory
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2007.12.008

In this paper, we present a method for human action recognition from multi-view image sequences that uses the combined motion and shape flow information with variability consideration. A combined local–global (CLG) optic flow is used to extract motion flow feature and invariant moments with flow deviations are used to extract the global shape flow feature from the image sequences. In our approach, human action is represented as a set of multidimensional CLG optic flow and shape flow feature vectors in the spatial–temporal action boundary. Actions are modeled by using a set of multidimensional HMMs for multiple views using the combined features, which enforce robust view-invariant operation. We recognize different human actions in daily life successfully in the indoor and outdoor environment using the maximum likelihood estimation approach. The results suggest robustness of the proposed method with respect to multiple views action recognition, scale and phase variations, and invariant analysis of silhouettes.