Detecting Deepfakes Without Seeing Any (2311.01458v1)

Abstract: Deepfake attacks, malicious manipulation of media containing people, are a serious concern for society. Conventional deepfake detection methods train supervised classifiers to distinguish real media from previously encountered deepfakes. Such techniques can only detect deepfakes similar to those previously seen, but not zero-day (previously unseen) attack types. As current deepfake generation techniques are changing at a breathtaking pace, new attack types are proposed frequently, making this a major issue. Our main observations are that: i) in many effective deepfake attacks, the fake media must be accompanied by false facts i.e. claims about the identity, speech, motion, or appearance of the person. For instance, when impersonating Obama, the attacker explicitly or implicitly claims that the fake media show Obama; ii) current generative techniques cannot perfectly synthesize the false facts claimed by the attacker. We therefore introduce the concept of "fact checking", adapted from fake news detection, for detecting zero-day deepfake attacks. Fact checking verifies that the claimed facts (e.g. identity is Obama), agree with the observed media (e.g. is the face really Obama's?), and thus can differentiate between real and fake media. Consequently, we introduce FACTOR, a practical recipe for deepfake fact checking and demonstrate its power in critical attack settings: face swapping and audio-visual synthesis. Although it is training-free, relies exclusively on off-the-shelf features, is very easy to implement, and does not see any deepfakes, it achieves better than state-of-the-art accuracy.

Analysis of "Detecting Deepfakes Without Seeing Any"

The increasing sophistication and prevalence of deepfake technology present a formidable challenge to existing detection methods. The paper "Detecting Deepfakes Without Seeing Any" by Reiss, Cavia, and Hoshen proposes a novel approach that departs from traditional supervised learning frameworks. Through the introduction of fact checking in the context of deepfake detection, this work addresses the persistent challenge of detecting zero-day attacks, a capability beyond the reach of many existing technologies.

Core Contributions

The paper introduces FACTOR, a methodology that leverages the discrepancies between claimed facts and their imperfect reproductions by generative models. At its heart, FACTOR reframes deepfake detection as a problem of verifying false or inaccurate claims associated with manipulated media. This approach is articulated through a general strategy that applies fact checking across several manifestations of deepfake media, including face swapping, audio-visual synthesis, and text-to-image generation.

1. Face Swapping Detection: Traditional methods falter when generalized to zero-day attacks due to reliance on previously seen data. FACTOR employs pre-trained face recognition features to verify claimed identities against real-world reference sets. It excels in zero-day scenarios, outperforming supervised baselines on datasets like DFDC and Celeb-DF by exploiting inherent discrepancies in facial identity synthesis.
2. Audio-Visual Deepfake Detection: Audio-visual manipulation detection leverages synchronization cues between audio and video streams. FACTOR applies pre-trained audio-visual features to estimate truth scores, which flag mismatches between claimed events in multimedia presentations.
3. Text-to-Image Deepfake Detection: By questioning the assumption that generative models can perfectly align synthetic images with textual prompts, the methodology highlights the overfitting of models like Stable Diffusion to their training paradigms. FACTOR uses the observed divergence between real and generated content to distinguish fakes, utilizing differences in correlation strength between CLIP and BLIP2 representations to achieve high ROC-AUC scores in the COCO dataset.

Implications and Modelling Insights

The proposed FACTOR framework offers several promising insights for future research and practical applications in the field of deepfake detection:

• Generalization to Unseen Attacks: By detaching from the need for fake data training, this method introduces a new standard for robustness in zero-day deepfake detection. Its reliance on true fact models ensures resilience against the fast-evolving landscape of deepfake generation technologies.
• Universal Applicability with Off-the-Shelf Features: The approach's deployment of off-the-shelf feature encoders illustrates a pathway for incremental development and improvement, leveraging advancements in related domains without committing extensive resources to dataset-specific training.
• Limitations and Expansion: The method naturally extends to various media types, yet it necessitates falsifiable facts accompanying media. Unconditional synthesized media present challenges, motivating additional research into integrating fact-checking principles where explicit claims are absent or implicit.

Conclusion

The authors present a compelling argument for adopting fact-based verification techniques in deepfake detection. This paper positions itself at the forefront of research striving to widen the scope of defenses against manipulation technologies. By focusing on the impossible perfection of generative models, the proposed FACTOR shifts detection away from recognizing known artifacts to understanding and leveraging the intrinsic limitations of current-generation synthesis methods. This approach not only offers significant performance improvements in critical scenarios but also encourages exploration into novel detection strategies resilient to future advancements in generative AI.