An empirical study of the impact of masks on face recognition

• Published in: Pattern recognition 2022-02, Vol.122, p.108308, Article 108308
• Main Authors: Jeevan, Govind, Zacharias, Geevar C., Nair, Madhu S., Rajan, Jeny
• Format: Article
• Language: English
• Subjects: Convolutional neural networks; COVID-19; Face recognition; Masked face
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2021.108308

•Presence of masks on faces degrade the performance of existing models trained on normal images.•Training with both normal and masked faces boosts masked face recognition performance.•Performance gap between deep and shallow models are wider in masked face recognition.•Optimal hyper-parameters for masked face recognition differ from normal face recognition.•CosFace loss function produces the best results on masked faces.•Real-world performance is reliant on the diversity of masks in training set. Face recognition has a wide range of applications like video surveillance, security, access control, etc. Over the past decade, the field of face recognition has matured and grown at par with the latest advancements in technology, particularly deep learning. Convolution Neural Networks have surpassed human accuracy in Face Recognition on popular evaluation tests such as LFW. However, most existing models evaluate their performance with an assumption of the availability of full facial information. The COVID-19 pandemic has laid forth challenges to this assumption, and to the performance of existing methods and leading-edge algorithms in the field of face recognition. This is in the wake of an explosive increase in the number of people wearing face masks. The reduced amount of facial information available to a recognition system from a masked face impacts their discrimination ability. In this context, we design and conduct a series of experiments comparing the masked face recognition performances of CNN architectures available in literature and exploring possible alterations in loss functions, architectures, and training methods that can enable existing methods to fully extract and leverage the limited facial information available in a masked face. We evaluate existing CNN-based face recognition systems for their performance against datasets composed entirely of masked faces, in contrast to the existing standard evaluations where masked or occluded faces are a rare occurrence. The study also presents evidence denoting an increased impact of network depth on performance compared to standard face recognition. Our observations indicate that substantial performance gains can be achieved by the introduction of masked faces in the training set. The study also inferred that various parameter settings determined suitable for standard face recognition are not ideal for masked face recognition. Through empirical analysis we derived new value recommendations for these paramete