A new framework of action recognition with discriminative parts, spatio-temporal and causal interaction descriptors

• Published in: Journal of visual communication and image representation 2018-10, Vol.56, p.116-130
• Main Authors: Tong, Ming, Chen, Yiran, Zhao, Mengao, Tian, Weijuan
• Format: Article
• Language: English
• Subjects: Action part; Action recognition; Causal relationship; Discriminative constraint; Spatio-temporal relationship; Spectral clustering
• ISSN: 1047-3203; 1095-9076
• DOI: 10.1016/j.jvcir.2018.09.001

•A discriminative spectral clustering method is proposed.•Action parts with the better discriminativeness are mined.•A spatio-temporal interaction descriptor is constructed.•A causal interaction descriptor is mined.•A new action recognition framework with better accuracy is presented. To improve action recognition performance, a novel discriminative spectral clustering method is firstly proposed, by which the candidate parts with the internal trajectories being close in spatial position, consistent in appearance and similar in motion velocity are mined. Furthermore, the discriminative constraint is introduced to select discriminative parts. Meanwhile, by fully considering the local and global distributions of data, a new similarity matrix is constructed, which enhances clustering effect. Secondly, the spatio-temporal interaction descriptor and causal interaction descriptor are constructed respectively, which fully mine the spatio-temporal and implicit causal interactive relationships between parts. Finally, a new framework is proposed. By associating the discriminative parts, spatio-temporal and causal interaction descriptors together as the inputs of Latent Support Vector Machine (LSVM), the correlations between action categories and action parts as well as interaction descriptors are mined. Consequently, accuracy is enhanced. The extensive and adequate experiments demonstrate the effectiveness of the proposed method.