Visible-infrared person re-identification using high utilization mismatch amending triplet loss

• Published in: Image and vision computing 2023-10, Vol.138, p.104797, Article 104797
• Main Authors: Zhu, Jianqing, Wu, Hanxiao, Zhao, Qianqian, Zeng, Huanqiang, Zhu, Xiaobin, Huang, Jingchang, Cai, Canhui
• Format: Article
• Language: English
• Subjects: Intelligent video surveillance; Modal-mismatch; Triplet loss; Visible-infrared person re-identification
• ISSN: 0262-8856
• DOI: 10.1016/j.imavis.2023.104797

Visible-infrared person re-identification (VIPR) is a task of retrieving a specific pedestrian monitored by cameras in different spectra. A dilemma of VIPR is how to reasonably use intra-modal pairs. Fully discarding intra-modal pairs causes a low utilization of training data, while using intra-modal pairs brings a danger of distracting a VIPR model's concentration on handling cross-modal pairs, harming the cross-modal similarity metric learning. For that, a high utilization mismatch amending (HUMA) triplet loss function is proposed for VIPR. The key of HUMA is the multi-modal matching regularization (MMMR), which restricts variations of distance matrices calculated from cross- and intra-modal pairs to cohere cross- and intra-modal similarity metrics, allowing for a high utilization of training data and amending the adverse distractions of intra-modal pairs. In addition, to avoid the risk of harming feature discrimination caused by MMMR preferring coherence in similarity metrics, a novel separated loss function assignment (SLFA) strategy is designed to arrange MMMR well. Experimental results show that the proposed method is superior to state-of-the-art approaches. •Intra-modal pairs are important for visible-infrared person re-identification.•Multi-modal matching regularization allows for a reasonable using intra-modal pairs.•Separated loss function assignment strategy preserves feature discrimination.