Decomposing Visual and Semantic Correlations For Both Fully Supervised and Few-Shot Image Classification

• Published in: IEEE transactions on artificial intelligence 2024-04, Vol.5 (4), p.1-11
• Main Authors: Zhang, Chunjie, Zheng, Xiaolong
• Format: Article
• Language: English
• Subjects: Classification and regression; Correlation; Image reconstruction; Machine learning; Matrix decomposition; Noise measurement; Semantic-guided artificial intelligence; Semantics; Testing; Visualization
• ISSN: 2691-4581; 2691-4581
• DOI: 10.1109/TAI.2023.3329457

Most image classification methods are designed to either boost the classification accuracies with abundant supervision, or cope with the shortage of supervision information. This is often achieved by using the visual and semantic information of other sources. However, these methods use the visual and semantic information within the same class independently, leaving their intrinsic correlations unconsidered. Objects and disturbing components as well as noise exist on the same image. Besides, semantic representations also contain noisy information. To solve the problems mentioned above, we propose a novel method for both fully supervised image classification and few-shot image classification by decomposing visual and semantic correlations (DVSC).We jointly explore the intrinsic correlations of visual and semantic information of images within the same class. For each class, we decompose its visual and semantic correlations using low-rank and sparse constraint respectively. The decomposed low-rank parts character the intrinsic correlations of images can be used in a linear transformation way. Using the decomposed parts of each class, Classification can be achieved by reconstruction error minimization. We conduct experiments on several datasets for both fully supervised image classification and few-shot image classification. Experimental results well show the effectiveness of the proposed method.