4-dimensional local spatio-temporal features for human activity recognition

• Main Authors: Hao Zhang, Parker, L. E.
• Format: Conference Proceeding
• Language: English
• Subjects: Cameras; Feature extraction; Humans; Three dimensional displays; Vectors; Videos
• ISSN: 2153-0858; 2153-0866
• DOI: 10.1109/IROS.2011.6094489

Recognizing human activities from common color image sequences faces many challenges, such as complex backgrounds, camera motion, and illumination changes. In this paper, we propose a new 4-dimensional (4D) local spatio-temporal feature that combines both intensity and depth information. The feature detector applies separate filters along the 3D spatial dimensions and the 1D temporal dimension to detect a feature point. The feature descriptor then computes and concatenates the intensity and depth gradients within a 4D hyper cuboid, which is centered at the detected feature point, as a feature. For recognizing human activities, Latent Dirichlet Allocation with Gibbs sampling is used as the classifier. Experiments are performed on a newly created database that contains six human activities, each with 33 samples with complex variations. Experimental results demonstrate the promising performance of the proposed features for the task of human activity recognition.