A progressive hierarchical analysis model for collective activity recognition

• Published in: Neural computing & applications 2022-08, Vol.34 (15), p.12415-12425
• Main Authors: Pei, Lishen, Zhao, Xuezhuan, Li, Tao, Zhang, Zheng
• Format: Article
• Language: English
• Subjects: Activity recognition; Algorithms; Artificial Intelligence; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Conditional random fields; Context; Data Mining and Knowledge Discovery; Deep learning; Image Processing and Computer Vision; Machine learning; Motion perception; Neural networks; Probability and Statistics in Computer Science; S.I: Machine Learning based semantic representation and analytics for multimedia application; Semantics; Special Issue on Machine Learning based semantic representation and analytics for multimedia application
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-021-06585-4

We propose a progressive hierarchical analysis model to perceive the collective activities. Compared with previous activity recognition works, it not only recognizes the collective activities, but also perceives the location and the action category of each individual. At first, we perform the person temporal consistency detection procedure for each individual of the collective activities. A person detection network and conditional random field are used to receive the bounding box sequences of the activity participators. Then, we recognize the individual actions using the learned spatial features and the motion features based on LSTM. At last, the combination of the recognized person-level action category vector, the scene context features and the interaction Context features are used to recognize the collective activities. We evaluate the proposed approach on benchmark collective activity datasets. Extensive experiments demonstrate the effects of the progressive hierarchical analysis model.