Improving generalization of image recognition with multi-branch generation network and contrastive learning

• Published in: Multimedia tools and applications 2023-07, Vol.82 (18), p.28367-28387
• Main Authors: Tan, Zhi, Teng, Zhaofei
• Format: Article
• Language: English
• Subjects: Accuracy; Coders; Computer Communication Networks; Computer networks; Computer Science; Data Structures and Information Theory; Datasets; Domains; Feature maps; Learning; Multimedia Information Systems; Performance enhancement; Regularization; Special Purpose and Application-Based Systems
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-023-14397-y

Aiming at the phenomenon that the recognition performance of the Image Recognition model trained by the source domain dataset decreases significantly after it is transplanted to the target domain dataset with different distribution, a domain generalization model based on multi-branch generation network and contrastive learning (MGNDG) is proposed, by generating data different from the source domain to simulate the target domain. Firstly, a multi-branch generation network is constructed, and the multi-scale feature maps of different fields of view are obtained by convolution encoder; Secondly, the feature diversification is realized through the self attention regularization layer, and then the images with different distributions are synthesized by the decoder, so that the effect of simulating the data distribution in the target domain can be achieved. Then contrastive learning and two-way content consistency strategies are introduced to ensure the effectiveness of the generated image. At the same time, the strategies strengthen information interaction and learn the invariant representation between samples and improve the ability of model identification. Finally, the adversarial training multi-branch generation network and backbone network continuously improve the generalization performance of the image recognition domain of the model. Compared with traditional methods, the average recognition accuracy of the proposed model in five classical digital recognition datasets is improved by nearly 3 % . The accuracy of classification on CIFAR 10-C dataset is improved by at least 2.2 % The experimental results show the feasibility of this model and significantly improve the performance of the domain generalization model of image recognition affected by domain shift.