Multi-semantic hypergraph neural network for effective few-shot learning

• Published in: Pattern recognition 2023-10, Vol.142, p.109677, Article 109677
• Main Authors: Chen, Hao, Li, Linyan, Hu, Fuyuan, Lyu, Fan, Zhao, Liuqing, Huang, Kaizhu, Feng, Wei, Xia, Zhenping
• Format: Article
• Language: English
• Subjects: Few-shot learning; Hypergraph; Multi-semantic learning; Orthogonal training
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2023.109677

•Multi-semantic hypergraph explore higher-order relationships among few samples.•Orthogonalized mapping function helps to obtain rich multi-semantic information.•Multi-semantic distribution information improve the rationality of hypergraphs.•Hypergraph and Multi-Semantic Distribution Information with node-edge message passing. Recently, Graph-based Few-Shot Learning (FSL) methods exhibit good generalization by mining relations among few samples with Graph Neural Networks. However, most Graph-based FSL methods consider only binary relations and ignore the multi-semantic information of the global context knowledge. We propose a framework of Multi-Semantic Hypergraph for FSL (MSH-FSL) to explore complex latent high-order multi-semantic relations among the few samples. By mining the complex relationship structure of multi-node and multi-semantics, more refined feature representation can be learned, which yields better classification robustness. Specifically, we first construct a novel Multi-Semantic Hypergraph by obtaining associated instances with different semantic features via orthogonal mapping. With the constructed hypergraph, we then develop the Hyergraph Neural Network along with a novel multi-generation hypergraph message passing so as to better leverage the complex latent semantic relations among samples. Finally, after a number of generations, the hyper-node representations embedded in the learned hypergraph become more accurate for obtaining few-shot prediction. In the 5-way 1-shot task of ResNet-12 on mini-Imagenet dataset, the multi-semantic hypergraph outperforms single-semantic graph by 3.1%, and with the proposed semantic-distribution message passing, the improvement can further reach 6.1%.