Sequential inter-hop graph convolution neural network (SIhGCN) for skeleton-based human action recognition

•A graph convolution model for skeleton-based action recognition is proposed.•Normalized Laplacian Matrix is utilized to encode the graph information.•An attention-based feature aggregation is proposed to extract the salient features.•The proposed method achieves better results than the baseline mod...

Full description

Saved in:
Bibliographic Details
Published in:Expert systems with applications 2022-06, Vol.195, p.116566, Article 116566
Main Authors: Setiawan, Feri, Yahya, Bernardo Nugroho, Chun, Seok-Ju, Lee, Seok-Lyong
Format: Article
Language:English
Subjects:
Action recognition
Artificial neural networks
Attention mechanism
Color imagery
Feature aggregation
Graph convolutional neural network
Human activity recognition
Human motion
Invariants
Neural networks
Normalized Laplacian matrix
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•A graph convolution model for skeleton-based action recognition is proposed.•Normalized Laplacian Matrix is utilized to encode the graph information.•An attention-based feature aggregation is proposed to extract the salient features.•The proposed method achieves better results than the baseline models. Skeleton-based human action recognition has attracted a lot of attention due to its capability and potential to provide more information than just using the sequence of RGB images. The use of Graph Convolutional Neural Network (GCN) becomes more popular since it can model the human skeleton very well. However, the existing GCN architectures ignore the different levels of importance on each hop during the feature aggregation and use the final hop information for further calculation, resulting in considerable information loss. Besides, they use the standard Laplacian or adjacency matrix to encode the property of a graph into a set of vectors which has a limitation in terms of graph invariants. In this work, we propose a Sequential Inter-hop Graph Convolution Neural Network (SIhGCN) which can capture salient graph information from every single hop rather than the final hop only and our work utilizes the normalized Laplacian matrix which provides better representation since it relates well to graph invariants. The proposed method is validated on two large datasets, NTU-RBG + D and Kinetics, to demonstrate the superiority of our proposed method.
ISSN:0957-4174
1873-6793
DOI:10.1016/j.eswa.2022.116566