Interactive Two-Stream Network across Modalities for Deepfake Detection

As face forgery techniques have become more mature, the proliferation of deepfakes may threaten the security of human society. Although existing deepfake detection methods achieve good performance for in-dataset evaluation, it remains to be improved in the generalization ability, where the represent...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on circuits and systems for video technology 2023-11, Vol.33 (11), p.1-1
Main Authors: Wu, Jianghao, Zhang, Baopeng, Li, Zhaoyang, Pang, Guilin, Teng, Zhu, Fan, Jianping
Format: Article
Language:English
Subjects:
Convolution
cross-modality learning
Datasets
Deception
deepfake detection
Deepfakes
Discrete cosine transform
Discrete cosine transforms
Embedding
Faces
Feature extraction
Forgery
Frequency-domain analysis
inconsistency representation
Modules
Performance evaluation
Representations
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As face forgery techniques have become more mature, the proliferation of deepfakes may threaten the security of human society. Although existing deepfake detection methods achieve good performance for in-dataset evaluation, it remains to be improved in the generalization ability, where the representation of the imperceptible artifacts plays a significant role. In this paper, we propose an Interactive Two-Stream Network (ITSNet) to explore the discriminant inconsistency representation from the perspective of cross-modality. In particular, the patch-wise Decomposable Discrete Cosine Transform (DDCT) is adopted to extract fine-grained high-frequency clues, and information from different modalities communicates with each other via a designed interaction module. To perceive the temporal inconsistency, we first develop a Short-term Embedding Module (SEM) to refine subtle local inconsistency representation between adjacent frames, and then a Long-term Embedding Module (LEM) is designed to further refine the erratic temporal inconsistency representation from the long-range perspective. Extensive experimental results conducted on three public datasets show that ITSNet outperforms the state-of-the-art methods both in terms of in-dataset and cross-dataset evaluations.
ISSN:1051-8215
1558-2205
DOI:10.1109/TCSVT.2023.3269841