Hierarchical Space-Time Model Enabling Efficient Search for Human Actions

We propose a five-layer hierarchical space-time model (HSTM) for representing and searching human actions in videos. From a features point of view, both invariance and selectivity are desirable characteristics, which seem to contradict each other. To make these characteristics coexist, we introduce...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on circuits and systems for video technology 2009-06, Vol.19 (6), p.808-820
Main Authors: Huazhong Ning, Han, T.X., Walther, D.B., Ming Liu, Huang, T.S.
Format: Article
Language:English
Subjects:
Action recognition
action search
Algorithms
Applied sciences
Brain modeling
Computer vision
Detection, estimation, filtering, equalization, prediction
Exact sciences and technology
hierarchical space-time model (HSTM)
histogram of Gabor Orientations (HIGO)
Histograms
Human
Humans
Image processing
Information, signal and communications theory
Invariance
Miscellaneous
Object recognition
Robustness
Searching
Selectivity
Signal and communications theory
Signal processing
Signal, noise
Spatial databases
State of the art
Telecommunications and information theory
Testing
Turning
Video sequences
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a five-layer hierarchical space-time model (HSTM) for representing and searching human actions in videos. From a features point of view, both invariance and selectivity are desirable characteristics, which seem to contradict each other. To make these characteristics coexist, we introduce a coarse-to-fine search and verification scheme for action searching, based on the HSTM model. Because going through layers of the hierarchy corresponds to progressively turning the knob between invariance and selectivity, this strategy enables search for human actions ranging from rapid movements of sports to subtle motions of facial expressions. The introduction of the Histogram of Gabor Orientations feature makes the searching for actions go smoothly across the hierarchical layers of the HSTM model. The efficient matching is achieved by applying integral histograms to compute the features in the top two layers. The HSTM model was tested on three selected challenging video sequences and on the KTH human action database. And it achieved improvement over other state-of-the-art algorithms. These promising results validate that the HSTM model is both selective and robust for searching human actions.
ISSN:1051-8215
1558-2205
DOI:10.1109/TCSVT.2009.2017399