Zero-shot action recognition by clustered representation with redundancy-free features

• Published in: Machine vision and applications 2023-11, Vol.34 (6), p.116, Article 116
• Main Authors: Xia, Limin, Wen, Xin
• Format: Article
• Language: English
• Subjects: Activity recognition; Communications Engineering; Computer Science; Datasets; Feature recognition; Generative adversarial networks; Image Processing and Computer Vision; Networks; Noise; Original Paper; Outliers (statistics); Pattern Recognition; Redundancy; Representations; Semantics; Vision systems
• ISSN: 0932-8092; 1432-1769
• DOI: 10.1007/s00138-023-01470-7

Zero-shot action recognition (ZSAR) is a practical and challenging issue, which compensates for the shortcomings of existing action recognition by being able to recognize those action classes that don’t have visual representation during training. However, existing zero-shot action recognition doesn’t focus on the fact that the generated features have many outliers, which harms the recognition. A new method for zero-shot action recognition is proposed, which suppresses this defect by clustered representation with redundancy-free features. In addition, a generative adversarial network (GAN) with gradient penalty is trained to synthesize stable features, solving the problem of data imbalance and alleviating the bottleneck of unstable features generated in existing methods. To reduce the dimension and the subsequent computation, a redundancy-free feature is introduced into the ZSAR. Experiments performed on Olympic Sports, HMDB51, and UCF101 public datasets prove that our method outperforms the state-of-the-art approaches with absolute gains of 1.8%, 0.3%, and 1.7%, respectively, in zero-shot action recognition.