A Spatio-Temporal Appearance Representation for Video-Based Pedestrian Re-Identification

• Main Authors: Kan Liu, Bingpeng Ma, Wei Zhang, Rui Huang
• Format: Conference Proceeding
• Language: English
• Subjects: Adaptation models; Feature extraction; Image color analysis; Legged locomotion; Measurement; Training; Video sequences
• ISSN: 2380-7504
• DOI: 10.1109/ICCV.2015.434

Pedestrian re-identification is a difficult problem due to the large variations in a person's appearance caused by different poses and viewpoints, illumination changes, and occlusions. Spatial alignment is commonly used to address these issues by treating the appearance of different body parts independently. However, a body part can also appear differently during different phases of an action. In this paper we consider the temporal alignment problem, in addition to the spatial one, and propose a new approach that takes the video of a walking person as input and builds a spatio-temporal appearance representation for pedestrian re-identification. Particularly, given a video sequence we exploit the periodicity exhibited by a walking person to generate a spatio-temporal body-action model, which consists of a series of body-action units corresponding to certain action primitives of certain body parts. Fisher vectors are learned and extracted from individual body-action units and concatenated into the final representation of the walking person. Unlike previous spatio-temporal features that only take into account local dynamic appearance information, our representation aligns the spatio-temporal appearance of a pedestrian globally. Extensive experiments on public datasets show the effectiveness of our approach compared with the state of the art.