Supervised Spatio-Temporal Neighborhood Topology Learning for Action Recognition

• Published in: IEEE transactions on circuits and systems for video technology 2013-08, Vol.23 (8), p.1447-1460
• Main Authors: Ma, Andy J., Yuen, Pong C., Zou, Wilman W. W., Jian-Huang Lai
• Format: Article
• Language: English
• Subjects: Action recognition; Applied sciences; Context; Detection, estimation, filtering, equalization, prediction; Eigenvalues and eigenfunctions; Exact sciences and technology; Feature extraction; Image processing; Information, signal and communications theory; manifold learning; Manifolds; neighborhood topology learning; Pattern recognition; Signal and communications theory; Signal processing; Signal, noise; supervised spatial; Telecommunications and information theory; temporal pose correspondence; Topology; Vectors
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2013.2248494

Supervised manifold learning has been successfully applied to action recognition, in which class label information could improve the recognition performance. However, the learned manifold may not be able to well preserve both the local structure and global constraint of temporal labels in action sequences. To overcome this problem, this paper proposes a new supervised manifold learning algorithm called supervised spatio-temporal neighborhood topology learning (SSTNTL) for action recognition. By analyzing the topological characteristics in the context of action recognition, we propose to construct the neighborhood topology using both supervised spatial and temporal pose correspondence information. Employing the property in locality preserving projection (LPP), SSTNTL solves the generalized eigenvalue problem to obtain the best projections that not only separates data points from different classes, but also preserves local structures and temporal pose correspondence of sequences from the same class. Experimental results demonstrate that SSTNTL outperforms the manifold embedding methods with other topologies or local discriminant information. Moreover, compared with state-of-the-art action recognition algorithms, SSTNTL gives convincing performance for both human and gesture action recognition.