UGaitNet: Multimodal Gait Recognition With Missing Input Modalities

Gait recognition systems typically rely solely on silhouettes for extracting gait signatures. Nevertheless, these approaches struggle with changes in body shape and dynamic backgrounds; a problem that can be alleviated by learning from multiple modalities. However, in many real-life systems some mod...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on information forensics and security 2021-01, Vol.16, p.5452-5462
Main Authors: Marin-Jimenez, Manuel J., Castro, Francisco M., Delgado-Escano, Ruben, Kalogeiton, Vicky, Guil, Nicolas
Format: Article
Language:English
Subjects:
Artificial Intelligence
biometrics
Biometrics (access control)
Cameras
Computer architecture
Computer Science
Computer Vision and Pattern Recognition
Deep learning
Flow mapping
Gait
Gait recognition
multimodal
Optical flow (image analysis)
Optical imaging
Optical sensors
Training data
Videos
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Gait recognition systems typically rely solely on silhouettes for extracting gait signatures. Nevertheless, these approaches struggle with changes in body shape and dynamic backgrounds; a problem that can be alleviated by learning from multiple modalities. However, in many real-life systems some modalities can be missing, and therefore most existing multimodal frameworks fail to cope with missing modalities. To tackle this problem, in this work, we propose UGaitNet, a unifying framework for gait recognition, robust to missing modalities. UGaitNet handles and mingles various types and combinations of input modalities, i.e. pixel gray value, optical flow, depth maps, and silhouettes, while being camera agnostic. We evaluate UGaitNet on two public datasets for gait recognition: CASIA-B and TUM-GAID, and show that it obtains compact and state-of-the-art gait descriptors when leveraging multiple or missing modalities. Finally, we show that UGaitNet with optical flow and grayscale inputs achieves almost perfect (98.9%) recognition accuracy on CASIA-B (same-view "normal") and 100% on TUM-GAID ("ellapsed time"). Code will be available at https://github.com/avagait/ugaitnet
ISSN:1556-6013
1556-6021
DOI:10.1109/TIFS.2021.3132579