Deepfakes Audit Study

Updated 2 May 2026

Deepfakes are synthetic media generated by deep learning techniques that manipulate visual, audio, and data artifacts to mimic authentic content.
Audit studies employ controlled user experiments, technical benchmarks, and adversarial simulations to assess detection accuracy and system vulnerabilities.
Findings reveal critical detection gaps, with human and automated methods struggling against realistic deepfakes, impacting biometric security and policy enforcement.

Deepfakes—synthetic media generated or modified by deep learning systems to mimic authentic visual, audio, or data artifacts—have rapidly become a focus of audit studies due to their implications for media integrity, biometric security, disinformation, and platform policy enforcement. Audit studies in this domain evaluate the generation, detectability, and societal or operational impacts of deepfakes by deploying simulation, user studies, technical benchmarks, or field experiments. This article synthesizes the methodologies, principal findings, and open problems in audit studies of deepfakes across visual, audio, and data modalities.

1. Taxonomy of Deepfake Audit Targets and Modalities

Audit studies of deepfakes span multiple content types:

Visual Deepfakes: Include identity swap, attribute manipulation, expression swap, and entire face synthesis using autoencoders and GANs. Typical detection targets are photorealistic face-swaps, GAN-synthesized images, and facial reenactments (CH et al., 2024).
Audio Deepfakes: Synthesize or convert speaker voice signatures using TTS or voice conversion, targeting speaker identity spoofing and natural prosodic cues (Mai et al., 2023, Korshunov et al., 2023).
Audio-Visual Deepfakes: Integrate both visual and audio manipulation for synchronized lips and voice, directly addressing vulnerabilities in multimodal authentication and perception (Korshunov et al., 2023).
Text/Data Deepfakes: Emerge in financial auditing, where synthetic transactional records are generated to evade traditional anomaly detection in enterprise resource planning systems (Schreyer et al., 2019).

Table 1: Deepfake Modalities in Audit Studies

Modality	Core Audit Focus	Representative Papers
Image (face)	Human detection, algorithmic detection	(Bray et al., 2022, CH et al., 2024)
Video (face, political)	Human discernment, detection model robustness, credibility	(Groh et al., 2022, Wegmann et al., 14 Mar 2026, Chandra et al., 4 Mar 2025)
Audio (speech)	Speaker authentication, human detection, spoofing	(Mai et al., 2023, Korshunov et al., 2023)
Audio-Visual (AV sync)	Biometric spoofing, lip-sync, perception studies	(Korshunov et al., 2023, Wegmann et al., 14 Mar 2026)
Tabular data (accounting)	Audit evasion, adversarial generative attacks	(Schreyer et al., 2019)
NCIM (non-consensual media)	Platform takedown efficacy	(Qiwei et al., 2024)

Audit studies integrate these modalities to evaluate detection thresholds, human-auditor performance, model vulnerabilities, and platform response.

2. Methodological Approaches in Deepfake Audit Studies

Audit methodologies encompass controlled user experiments, adversarial benchmarking, field deployment, and dataset curation:

Controlled User Studies: Human subjects classify media items as real/fake under conditions varying in modality, cue salience, and intervention. Metrics include accuracy, d′ (signal detection), confidence calibration, and learning dynamics (Bray et al., 2022, Groh et al., 2022, Mai et al., 2023, Wegmann et al., 14 Mar 2026).
- Example: Bray et al. show humans' mean accuracy in image deepfake detection is 62.2%, with little effect from interventions (Bray et al., 2022).
- Studies on speech deepfakes report 70% overall accuracy, with no significant language effect, and negligible learning without feedback (Mai et al., 2023).
Technical Benchmarking: Evaluation of automated detection models on standard and in-the-wild datasets. Performance is compared using accuracy, AUC, precision, recall, and EER (CH et al., 2024, Chandra et al., 4 Mar 2025).
- Deepfake-Eval-2024 demonstrates that state-of-the-art detectors suffer 45–50% AUC drops on real-world data compared to academic benchmarks (Chandra et al., 4 Mar 2025).
Vulnerability Assessment: Spoofing success of biometric systems is measured via attack presentation match rates (e.g., IAPMR >90% for tuned audio-visual deepfakes) (Korshunov et al., 2023).
Platform Audits (Operational Field Experiments): Empirical tests of social platform response mechanisms to deepfake reporting (e.g., takedown rates for non-consensual intimate media). For instance, X (formerly Twitter) rapidly removes NCIM when reported as copyright infringement (100% in <25h) but not for privacy or nudity violations (0% in >3 weeks) (Qiwei et al., 2024).
Adversarial Simulation: Use of generative models to create realistic “fake” entries in financial data to evade anomaly detection, with assessment of CAAT evasion success (Schreyer et al., 2019).

3. Human and Machine Detection Audits: Performance, Limits, and Training

Human auditors in deepfake studies consistently underperform compared to ensemble judgments or domain-specific forensics:

Images: Bray et al. reported that lay users achieved only 62% accuracy at distinguishing StyleGAN2-generated faces from FFHQ real faces; interventions like feature advice or familiarization did not significantly improve results and caused overconfidence without true gain (Bray et al., 2022).
Audiovisual Content: In the context of political deepfakes, video+audio modalities enabled the highest discernment (up to 86%), outperforming transcript (57%) and audio-only (81%) (Groh et al., 2022). Fine-grained results demonstrate significant modality and cue source effects.
Speech: For speech deepfakes (TTS-synthesized), users identified fakes with ~70% accuracy; exposure to reference (genuine) audio raises accuracy to 85.6%, but practical deployment of such binary comparisons is limited (Mai et al., 2023).

Training, either by familiarization with fakes or by offering textual/visual cues to spotting artifacts, produces statistically detectable but modest effects (improvements ~3–5% absolute), and increases false positives as much as true positives (Bray et al., 2022, Mai et al., 2023). No significant learning occurs without trial-by-trial feedback.

Collective or “crowd” aggregation of judgments can boost AUC substantially (up to ~95%), but expert forensic analysts remain more consistent (Mai et al., 2023).

4. Deepfake Generation Methods and Adversarial Capabilities

Audit studies delineate four generative paradigms: attribute manipulation, expression swap (reenactment), identity swap (replacement), and entire face synthesis, adopting architectures such as autoencoders, variational autoencoders (VAEs), and GANs (DCGAN, StyleGAN, StarGAN) (CH et al., 2024, Zobaed et al., 2021).

Cybersecurity- and forensics-focused audit reports stress adversarial learning’s capacity to evade automated and human auditing:

In biometric security, face and speaker recognition systems are highly vulnerable to identity-targeted deepfakes: SWAN-DF deepfakes achieve IAPMR >97% for faces and >94% (tuned VC) for voices; combined audio-visual attacks yield attack acceptance rates >91% (Korshunov et al., 2023).
In tabular audit domains, adversarial autoencoders are shown to generate synthetic journal entries that evade both rule-based and statistical CAATs, with well-controlled semantic/attribute manipulation imitating authentic latent distributions (Schreyer et al., 2019).

5. Detection Audits: Datasets, Benchmarks, and Real-World Performance

Audit studies have surfaced critical gaps between laboratory detection performance and real-world robustness:

Benchmark Datasets: Academic datasets (FaceForensics++, Celeb-DF, DFDC, UADFV) facilitate high-accuracy detectors but lack real-world distribution shifts, newer manipulation techniques (e.g., diffusion models), multi-language/audio content, and multimodal anomalies (Zobaed et al., 2021, CH et al., 2024, Chandra et al., 4 Mar 2025).
- Deepfake-Eval-2024 addresses this by curating a 45 h video, 56.5 h audio, and 1,975 image benchmark from 2024’s online ecosystem, revealing open-source detector AUCs drop by 45–50% on these data (Chandra et al., 4 Mar 2025).
Detection Models: State-of-the-art models (MesoNet, XceptionNet, CNN+LSTM hybrids, RawNet2, AASIST) perform with >97% accuracy on academic data, but underperform by 10–20 AUC points compared to human forensic analysts on in-the-wild content (Chandra et al., 4 Mar 2025, CH et al., 2024). Human–AI ensembles and commercial models do better but still lag for challenging manipulations (e.g., silent audio, selective edits, non-English content).
Error Analysis: Domain shift (novel GANs/diffusion models), content drift (background text/audio, non-central faces), and silent or off-distribution input drive detector failures (Chandra et al., 4 Mar 2025). Adversarial attacks targeting model artifacts or training domain boundaries remain a practical threat (Zobaed et al., 2021).