Video Person Re-Identification with Frame Sampling-Random Erasure and Mutual Information-Temporal Weight Aggregation

• Published in: Sensors (Basel, Switzerland) Switzerland), 2022-04, Vol.22 (8), p.3047
• Main Authors: Li, Jiayue, Piao, Yan
• Format: Article
• Language: English
• Subjects: Aggregates; Cameras; Clutter; Deep learning; frame sampling–random erasure (FSE); Humans; Identification; Methods; mutual information–temporal weight aggregation (MI–TWA); Noise; Occlusion; Sampling; Surveillance; video person re-identification; Video Recording - methods
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s22083047

Partial occlusion and background clutter in camera video surveillance affect the accuracy of video-based person re-identification (re-ID). To address these problems, we propose a person re-ID method based on random erasure of frame sampling and temporal weight aggregation of mutual information of partial and global features. First, for the case in which the target person is interfered or partially occluded, the frame sampling-random erasure (FSE) method is used for data enhancement to effectively alleviate the occlusion problem, improve the generalization ability of the model, and match persons more accurately. Second, to further improve the re-ID accuracy of video-based persons and learn more discriminative feature representations, we use a ResNet-50 network to extract global and partial features and fuse these features to obtain frame-level features. In the time dimension, based on a mutual information-temporal weight aggregation (MI-TWA) module, the partial features are added according to different weights and the global features are added according to equal weights and connected to output sequence features. The proposed method is extensively experimented on three public video datasets, MARS, DukeMTMC-VideoReID, and PRID-2011; the mean average precision (mAP) values are 82.4%, 94.1%, and 95.3% and Rank-1 values are 86.4%, 94.8%, and 95.2%, respectively.