Visible-Infrared Person Re-Identification via Homogeneous Augmented Tri-Modal Learning

• Published in: IEEE transactions on information forensics and security 2021, Vol.16, p.728-739
• Main Authors: Ye, Mang, Shen, Jianbing, Shao, Ling
• Format: Article
• Language: English
• Subjects: Cameras; Classification; Face recognition; Gray scale; Image color analysis; Infrared imagery; Machine learning; multi-modality; Parameter identification; Person re-identification (Re-ID); ranking; Retrieval; Surveillance; Task analysis; Training
• ISSN: 1556-6013; 1556-6021
• DOI: 10.1109/TIFS.2020.3001665

Matching person images between the daytime visible modality and night-time infrared modality (VI-ReID) is a challenging cross-modality pedestrian retrieval problem. Existing methods usually learn the multi-modality features in raw image, ignoring the image-level discrepancy. Some methods apply GAN technique to generate the cross-modality images, but it destroys the local structure and introduces unavoidable noise. In this paper, we propose a Homogeneous Augmented Tri-Modal (HAT) learning method for VI-ReID, where an auxiliary grayscale modality is generated from their homogeneous visible images, without additional training process. It preserves the structure information of visible images and approximates the image style of infrared modality. Learning with the grayscale visible images enforces the network to mine structure relations across multiple modalities, making it robust to color variations. Specifically, we solve the tri-modal feature learning from both multi-modal classification and multi-view retrieval perspectives. For multi-modal classification, we learn a multi-modality sharing identity classifier with a parameter-sharing network, trained with a homogeneous and heterogeneous identification loss. For multi-view retrieval, we develop a weighted tri-directional ranking loss to optimize the relative distance across multiple modalities. Incorporated with two invariant regularizers, HAT simultaneously minimizes multiple modality variations. In-depth analysis demonstrates the homogeneous grayscale augmentation significantly outperforms the current state-of-the-art by a large margin.