Multimodal Gait Recognition Under Missing Modalities

Ruben Delgado-Escano (1) , Francisco M Castro (1) , Nicolas Guil (1) , Vicky Kalogeiton (2) , Manuel J Marin-Jimenez (3)

1 Universidad de Málaga [Málaga] = University of Málaga [Málaga]
2 LIX - Laboratoire d'informatique de l'École polytechnique [Palaiseau]
3 Universidad de Córdoba = University of Córdoba [Córdoba]

Multimodal systems for gait recognition have gained a lot of attention. However, there is a clear gap in the study of missing modalities, which represents real-life scenarios where sensors fail or data get corrupted. Here, we investigate how to handle missing modalities for gait recognition. We propose a single and flexible framework that uses a variable number of input modalities. For each modality, it consists of a branch and a binary unit indicating whether the modality is available; these are gated and merged together. Finally, it generates a single and compact 'multimodal' gait signature that encodes biometric information of the input. Our framework outperforms the state of the art on TUM-GAID and extensive experiments reveal its effectiveness for handling missing modalities even in the multiview setup of CASIA-B. The code is available online: https://github.com/avagait/gaitmiss.

Domains: Computer Science [cs]; Computer Vision and Pattern Recognition [cs.CV]; Computer Science [cs]; Artificial Intelligence [cs.AI]

Ruben Delgado-Escano, Francisco M Castro, Nicolas Guil, Vicky Kalogeiton, Manuel J Marin-Jimenez. Multimodal Gait Recognition Under Missing Modalities. 2021 IEEE International Conference on Image Processing (ICIP), Sep 2021, Anchorage, Alaska (virtual), United States. ⟨10.1109/ICIP42928.2021.9506162⟩. ⟨hal-03353572⟩