Multimodal Gait Recognition Under Missing Modalities

Multimodal systems for gait recognition have gained a lot of attention. However, there is a clear gap in the study of missing modalities, which represents real-life scenarios where sensors fail or data get corrupted. Here, we investigate how to handle missing modalities for gait recognition. We prop...

Full description

Saved in:
Bibliographic Details
Main Authors: Delgado-Escano, Ruben, Castro, Francisco M., Guil, Nicolas, Kalogeiton, Vicky, Marin-Jimenez, Manuel J.
Format: Conference Proceeding
Language:English
Subjects:
Conferences
Gait recognition
Image processing
Logic gates
Optical flow
Sensors
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Multimodal systems for gait recognition have gained a lot of attention. However, there is a clear gap in the study of missing modalities, which represents real-life scenarios where sensors fail or data get corrupted. Here, we investigate how to handle missing modalities for gait recognition. We propose a single and flexible framework that uses a variable number of input modalities. For each modality, it consists of a branch and a binary unit indicating whether the modality is available; these are gated and merged together. Finally, it generates a single and compact 'multimodal' gait signature that encodes biometric information of the input. Our framework outperforms the state of the art on TUM-GAID and extensive experiments reveal its effectiveness for handling missing modalities even in the multiview setup of CASIA-B. The code is available online: https://github.com/avagait/gaitmiss.
ISSN:2381-8549
DOI:10.1109/ICIP42928.2021.9506162