Augmentation Invariant and Instance Spreading Feature for Softmax Embedding

• Published in: IEEE transactions on pattern analysis and machine intelligence 2022-02, Vol.44 (2), p.924-939
• Main Authors: Ye, Mang, Shen, Jianbing, Zhang, Xu, Yuen, Pong C., Chang, Shih-Fu
• Format: Article
• Language: English
• Subjects: Algorithms; Attention; Categories; Data augmentation; Data mining; Embedding; embedding learning; instance feature; Invariants; Optimization; Representations; softmax embedding; Supervised learning; Supervision; Task analysis; Testing; Training; Unsupervised learning; Visualization
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2020.3013379

Deep embedding learning plays a key role in learning discriminative feature representations, where the visually similar samples are pulled closer and dissimilar samples are pushed away in the low-dimensional embedding space. This paper studies the unsupervised embedding learning problem by learning such a representation without using any category labels. This task faces two primary challenges: mining reliable positive supervision from highly similar fine-grained classes, and generalizing to unseen testing categories. To approximate the positive concentration and negative separation properties in category-wise supervised learning, we introduce a data augmentation invariant and instance spreading feature using the instance-wise supervision. We also design two novel domain-agnostic augmentation strategies to further extend the supervision in feature space, which simulates the large batch training using a small batch size and the augmented features. To learn such a representation, we propose a novel instance-wise softmax embedding, which directly perform the optimization over the augmented instance features with the binary discrmination softmax encoding. It significantly accelerates the learning speed with much higher accuracy than existing methods, under both seen and unseen testing categories. The unsupervised embedding performs well even without pre-trained network over samples from fine-grained categories. We also develop a variant using category-wise supervision, namely category-wise softmax embedding, which achieves competitive performance over the state-of-of-the-arts, without using any auxiliary information or restrict sample mining.