Balanced and Essential Modality-Specific and Modality-Shared Representations for Visible-Infrared Person Re-Identification

• Published in: IEEE signal processing letters 2024, Vol.31, p.491-495
• Main Authors: Gwon, Soonyong, Kim, Sejun, Seo, Kisung
• Format: Article
• Language: English
• Subjects: balanced and essential modality-specific and modality-shared network; balanced modality-specific and modality-shared; Cameras; customized modality-specific enhanced module; Data augmentation; Extreme values; Feature extraction; Feature maps; Image color analysis; Indexes; informative weighted gray transform; Infrared imagery; Labels; pseudo label-oriented modality-specific loss; Representations; Task analysis; Training; Transforms; Visible-infrared person re-identification
• ISSN: 1070-9908; 1558-2361
• DOI: 10.1109/LSP.2024.3358725

Retrieving and matching individual images for Visible-Infrared Person Re-identification is a challenging task due to the huge modality gap between daytime color and nighttime infrared images from different modalities. Existing approaches rely on inefficient data augmentation and/or biased modality characteristics, limiting their potential for performance improvement. To solve these problems, we propose a novel balanced approach between Modality-Specific and Modality-Shared method including efficient transform for diversity of instances. First, we propose the Informative Weighted Gray transform (IWG), which aims to maximize the diversity of instances by generating the distinct combination of RGB colors. Second, introducing the Customized Modality-Specific Enhanced Module (CMSpEM) provides enhanced feature maps using attention mechanism between pre-pooling and post-pooling features and reinforces specific features for the modality-shared based network with an extremely small number of parameters. Third, we introduce the Pseudo Label-oriented Modality-Specific (PLMSp) Loss, which provides effective representation learning explicitly to reduce the modality gap using pseudo labels as anchors. We compare our Balanced and Essential Modality-Specific and Modality-Shared Network (BEMSSNet) and various existing methods for the mAP and Rank-1 performances on the SYSU-MM01 and RegDB datasets. Experimental results demonstrate that our proposed model outperforms the existing state-of-the-art methods.