TFRS: A task-level feature rectification and separation method for few-shot video action recognition

• Published in: Neural networks 2024-08, Vol.176, p.106326, Article 106326
• Main Authors: Qin, Yanfei, Liu, Baolin
• Format: Article
• Language: English
• Subjects: Algorithms; Feature rectification; Feature separation; Few-shot video action recognition; Humans; Pattern Recognition, Automated - methods
• ISSN: 0893-6080; 1879-2782; 1879-2782
• DOI: 10.1016/j.neunet.2024.106326

Few-shot video action recognition (FS-VAR) is a challenging task that requires models to have significant expressive power in order to identify previously unseen classes using only a few labeled examples. However, due to the limited number of support samples, the model’s performance is highly sensitive to the distribution of the sampled data. The representativeness of the support data is insufficient to cover the entire class, and the support features may contain shared information that confuses the classifier, leading to biased classification. In response to this difficulty, we present a task-level feature rectification and separation (TFRS) method that effectively resolves the sample bias issue. Our main idea is to leverage prior information from base classes to rectify the support samples while removing the commonality of task-level features. This enhances the distinguishability and separability of features in space. Furthermore, TFRS offers a straightforward yet versatile solution that can be seamlessly integrated into various established FS-VAR frameworks. Our design yields significant performance enhancements across various existing works by implementing TFRS, resulting in competitive outcomes on datasets such as UCF101, Kinetics, SSv2, and HMDB51. •We rectifies support sample by utilizing information from similar base prototypes.•We remove the projection onto the shared centroid.•Our method can be applied to various established FS-VAR frameworks.•We emphasize the importance of reducing the widely prevalent sample selection bias.