How Do Deepfakes Move? Motion Magnification for Deepfake Source Detection

• Main Authors: Demir, Ilke, Ciftci, Umur Aybars
• Format: Conference Proceeding
• Language: English
• Subjects: accountable; Algorithms; Biometrics; body pose; Buildings; Computational modeling; Computer architecture; Computer vision; Deepfakes; Detectors; ethical computer vision; Explainable; face; fair; gesture; privacy-preserving; Skin
• ISSN: 2642-9381
• DOI: 10.1109/WACV57701.2024.00471

With the proliferation of deep generative models, deepfakes are improving in quality and quantity everyday. However, there are subtle authenticity signals in pristine videos, not replicated by current generative models. We contrast the movement in deepfakes and authentic videos by motion magnification towards building a generalized deepfake source detector. The sub-muscular motion in faces has different interpretations per different generative models, which is reflected in their generative residue. Our approach exploits the difference between real motion and the amplified generative artifacts, by combining deep and traditional motion magnification, to detect whether a video is fake and its source generator if so. Evaluating our approach on two multi-source datasets, we obtain 97.77% and 94.03% for video source detection. Our approach performs at least 4.08% better than the prior deepfake source detector and other complex architectures. We also analyze magnification amount, phase extraction window, backbone network, sample counts, and sample lengths. Finally, we report our results on skin tones and genders to assess the model bias.