Improved likelihood ratios for face recognition in surveillance video by multimodal feature pairing

•Study tackles the problem of accurate likelihood ratios in surveillance video face recognition.•Proposes a frame selection method to enhance forensic face recognition by pairing frames based on quality and face attributes.•Optimal frame selection framework validated across various models and datase...

Full description

Saved in:
Bibliographic Details
Published in:Forensic science international. Synergy 2024-01, Vol.8, p.100458-100458, Article 100458
Main Authors: Macarulla Rodriguez, Andrea, Geradts, Zeno, Worring, Marcel, Unzueta, Luis
Format: Article
Language:English
Subjects:
Face image quality
Face recognition
Likelihood ratio
Multi-modal analysis
Super resolution
Video processing
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•Study tackles the problem of accurate likelihood ratios in surveillance video face recognition.•Proposes a frame selection method to enhance forensic face recognition by pairing frames based on quality and face attributes.•Optimal frame selection framework validated across various models and datasets.•As additional result, the study reveals that the use of super-resolution preprocessing via CodeFormer adversely affects the reliability of forensic face recognition. In forensic and security scenarios, accurate facial recognition in surveillance videos, often challenged by variations in pose, illumination, and expression, is essential. Traditional manual comparison methods lack standardization, revealing a critical gap in evidence reliability. We propose an enhanced images-to-video recognition approach, pairing facial images with attributes like pose and quality. Utilizing datasets such as ENFSI 2015, SCFace, XQLFW, ChokePoint, and ForenFace, we assess evidence strength using calibration methods for likelihood ratio estimation. Three models—ArcFace, FaceNet, and QMagFace—undergo validation, with the log-likelihood ratio cost (Cllr) as a key metric. Results indicate that prioritizing high-quality frames and aligning attributes with reference images optimizes recognition, yielding similar Cllr values to the top 25% best frames approach. A combined embedding weighted by frame quality emerges as the second-best method. Upon preprocessing facial images with the super resolution CodeFormer, it unexpectedly increased Cllr, undermining evidence reliability, advising against its use in such forensic applications.
ISSN:2589-871X
2589-871X
DOI:10.1016/j.fsisyn.2024.100458