Deep learning network model based on fusion of spatiotemporal features for action recognition

In view of the problem that the current deep learning network does not fully extract and fuse spatio-temporal information in the action recognition task, resulting in low recognition accuracy, this paper proposes a deep learning network model based on fusion of spatio-temporal features (FSTFN). Thro...

Full description

Saved in:
Bibliographic Details
Published in:Multimedia tools and applications 2022-03, Vol.81 (7), p.9875-9896
Main Authors: Yang, Ge, Zou, Wu-xing
Format: Article
Language:English
Subjects:
Accuracy
Activity recognition
Artificial neural networks
Computer Communication Networks
Computer Science
Data Structures and Information Theory
Datasets
Deep learning
Feature recognition
Machine learning
Multimedia Information Systems
Special Purpose and Application-Based Systems
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In view of the problem that the current deep learning network does not fully extract and fuse spatio-temporal information in the action recognition task, resulting in low recognition accuracy, this paper proposes a deep learning network model based on fusion of spatio-temporal features (FSTFN). Through two networks composed of CNN (Convolutional Neural Networks) and LSTM (Long Short-Term Memory), the time and space information are extracted and fused; multi-segment input is used to process large-scale video frame information to solve the problem of long-term dependence and improve the prediction accuracy; The attention mechanism improves the weight of visual subjects in the network. The experimental verification on the UCF101 (University of Central Florida 101) data set shows that the prediction accuracy of the proposed FSFTN on the UCF101 data set is 92.7%, 4.7% higher than that of Two-stream, which verifies the effectiveness of the network model.
ISSN:1380-7501
1573-7721
DOI:10.1007/s11042-022-11937-w