Hankelet-based dynamical systems modeling for 3D action recognition

• Published in: Image and vision computing 2015-12, Vol.44, p.29-43
• Main Authors: Lo Presti, Liliana, La Cascia, Marco, Sclaroff, Stan, Camps, Octavia
• Format: Article
• Language: English
• Subjects: Action; Discriminative learning; Hankel Matrix; Hidden Markov Model; Linear time invariant system
• ISSN: 0262-8856; 1872-8138
• DOI: 10.1016/j.imavis.2015.09.007

This paper proposes to model an action as the output of a sequence of atomic Linear Time Invariant (LTI) systems. The sequence of LTI systems generating the action is modeled as a Markov chain, where a Hidden Markov Model (HMM) is used to model the transition from one atomic LTI system to another. In turn, the LTI systems are represented in terms of their Hankel matrices. For classification purposes, the parameters of a set of HMMs (one for each action class) are learned via a discriminative approach. This work proposes a novel method to learn the atomic LTI systems from training data, and analyzes in detail the action representation in terms of a sequence of Hankel matrices. Extensive evaluation of the proposed approach on two publicly available datasets demonstrates that the proposed method attains state-of-the-art accuracy in action classification from the 3D locations of body joints (skeleton). [Display omitted] •We model an action as sequence of outputs of linear time invariant (LTI) systems.•We represent the outputs of LTI systems by means of Hankelets.•We adopt an HMM to model the transitions from one LTI system to another.•We formulate an inference and supervised learning formulation for our model.•We also present a deep analysis of the parameter settings for our action representation.