Action Recognition Using Nonnegative Action Component Representation and Sparse Basis Selection

• Published in: IEEE transactions on image processing 2014-02, Vol.23 (2), p.570-581
• Main Authors: Wang, Haoran, Yuan, Chunfeng, Hu, Weiming, Ling, Haibin, Yang, Wankou, Sun, Changyin
• Format: Article
• Language: English
• Subjects: action recognition; Action unit; Algorithms; Applied sciences; Context; Detection, estimation, filtering, equalization, prediction; Dictionaries; Exact sciences and technology; Feature extraction; Humans; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Image processing; Imaging, Three-Dimensional - methods; Information, signal and communications theory; Motor Activity - physiology; Movement - physiology; nonnegative matrix factorization; Pattern Recognition, Automated - methods; Photography - methods; Reproducibility of Results; Semantics; Sensitivity and Specificity; Signal and communications theory; Signal processing; Signal, noise; sparse representation; Subtraction Technique; Telecommunications and information theory; Vectors; Videos; Visualization; Whole Body Imaging - methods
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2013.2292550

In this paper, we propose using high-level action units to represent human actions in videos and, based on such units, a novel sparse model is developed for human action recognition. There are three interconnected components in our approach. First, we propose a new context-aware spatial-temporal descriptor, named locally weighted word context, to improve the discriminability of the traditionally used local spatial-temporal descriptors. Second, from the statistics of the context-aware descriptors, we learn action units using the graph regularized nonnegative matrix factorization, which leads to a part-based representation and encodes the geometrical information. These units effectively bridge the semantic gap in action recognition. Third, we propose a sparse model based on a joint l2,1-norm to preserve the representative items and suppress noise in the action units. Intuitively, when learning the dictionary for action representation, the sparse model captures the fact that actions from the same class share similar units. The proposed approach is evaluated on several publicly available data sets. The experimental results and analysis clearly demonstrate the effectiveness of the proposed approach.