Temporal Localization of Actions with Actoms

We address the problem of localizing actions, such as opening a door, in hours of challenging video data. We propose a model based on a sequence of atomic action units, termed "actoms," that are semantically meaningful and characteristic for the action. Our actom sequence model (ASM) repre...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2013-11, Vol.35 (11), p.2782-2795
Main Authors: Gaidon, A., Harchaoui, Z., Schmid, C.
Format: Article
Language:English
Subjects:
Actigraphy - methods
Action recognition
actoms
Adaptation models
Algorithms
Applied sciences
Artificial Intelligence
Computer Science
Computer science
control theory
systems
Computer Vision and Pattern Recognition
Exact sciences and technology
Hidden Markov models
Histograms
Humans
Image Interpretation, Computer-Assisted - methods
Pattern Recognition, Automated - methods
Pattern recognition. Digital image processing. Computational geometry
Spatiotemporal phenomena
Subtraction Technique
Support vector machines
temporal localization
Training
video analysis
Video Recording - methods
Visualization
Whole Body Imaging - methods
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We address the problem of localizing actions, such as opening a door, in hours of challenging video data. We propose a model based on a sequence of atomic action units, termed "actoms," that are semantically meaningful and characteristic for the action. Our actom sequence model (ASM) represents an action as a sequence of histograms of actom-anchored visual features, which can be seen as a temporally structured extension of the bag-of-features. Training requires the annotation of actoms for action examples. At test time, actoms are localized automatically based on a nonparametric model of the distribution of actoms, which also acts as a prior on an action's temporal structure. We present experimental results on two recent benchmarks for action localization "Coffee and Cigarettes" and the "DLSBP" dataset. We also adapt our approach to a classification-by-localization set-up and demonstrate its applicability on the challenging "Hollywood 2" dataset. We show that our ASM method outperforms the current state of the art in temporal action localization, as well as baselines that localize actions with a sliding window method.
ISSN:0162-8828
1939-3539
2160-9292
DOI:10.1109/TPAMI.2013.65