--- title: 'DDL-AV Dataset: Audio-Visual Deepfake Benchmark' url: https://www.emergentmind.com/topics/ddl-av-dataset type: topic --- # DDL-AV Dataset: Audio-Visual Deepfake Benchmark Searching arXiv for the DDL and DDL-AV papers to ground the article in the cited sources. DDL-AV is the audio-visual subset of the DDL benchmark for interpretable deepfake detection and localization in real-world scenarios. Within the broader DDL dataset, which contains over $1.8\text{M}$ forged samples and supports classification, temporal localization, and spatial localization, DDL-AV contributes more than $0.3\text{M}$ audio-video samples and is defined by multimodal forgery diversity, varied manipulation modes, and fine-grained temporal annotations for both audio and video streams [2506.23292]. It is also described as containing both segmented and full-length video clips, enabling evaluation beyond isolated short segments and toward more realistic deployment conditions [2508.17282]. ## 1. Position within the DDL benchmark DDL-AV is not an independent benchmark in isolation; it is the audio-visual component of the larger DDL dataset. The parent DDL resource is organized into an image unimodal subset, DDL-I, with more than $1.5\text{M}$ images, and an audio-visual subset, DDL-AV, with more than $0.3\text{M}$ audio-video samples covering audio, video, and audio-video forgeries [2506.23292]. The DDL paper frames this design around four key innovations: **Diverse Forgery Scenarios**, **Comprehensive Deepfake Methods**, **Varied Manipulation Modes**, and **Fine-grained Forgery Annotations**. For DDL as a whole, the benchmark draws on 14+ real datasets for source diversity, including FF++, Celeb-DF, DFDC, FFIW, CelebA, FFHQ, and CelebV-HQ. In the original-source statistics reported for the audio-video branch, the train/valid partition uses VoxCeleb2 (72K), while the test partition uses VoxCeleb2 (10K) and CelebV-HQ (15K), for a total of 97K source samples [2506.23292]. The distinction between 97K source samples and more than $0.3\text{M}$ DDL-AV samples indicates that the released benchmark is a generated and annotated deepfake corpus rather than a mere repackaging of raw source media. | Aspect | DDL-AV | |---|---| | Role in DDL | Audio-visual subset | | Scale | $>0.3\text{M}$ audio-video samples | | Source-data total | 97K original audio-video samples | | Primary source datasets | VoxCeleb2, CelebV-HQ | | Supported task types in DDL context | Detection and localization | ## 2. Data composition and modality coverage DDL-AV is explicitly multimodal. It contains both the audio and the visual channel, and its benchmark logic includes cases in which one modality is manipulated while the other remains authentic [2508.17282]. The DDL paper further specifies that DDL-AV covers audio, video, and audio-video samples, situating it within a broader program of multimodal deepfake detection and localization [2506.23292]. The benchmark is designed around realistic forgery variability rather than purely synchronized, fully manipulated clips. In the ERF-BA-TFD+ description, DDL-AV includes both segmented and full-length video clips, and each video can include both manipulated and non-manipulated segments [2508.17282]. This structure matters technically because it turns the task from simple clip-level binary classification into a joint detection-and-localization problem over long temporal contexts. Public descriptions emphasize modality coverage and annotation granularity more strongly than exhaustive clip-count statistics. The ERF-BA-TFD+ paper states that the dataset contains enough long-duration and full-length video samples to challenge detectors on temporally dispersed manipulations, but the exact number of clips, average length, and number of segments are not given verbatim there [2508.17282]. ## 3. Forgery scenarios and manipulation taxonomy DDL-AV is characterized by a broad forgery space spanning both audio and video. In the benchmark description, audio forgeries include state-of-the-art techniques such as **text-to-speech**, **voice cloning**, and **voice swapping**, while visual forgeries include **face swapping**, **facial animation**, and **text-to-video (AIGC) generation** [2508.17282]. This places DDL-AV in direct contact with both classical face-manipulation pipelines and newer generative-content workflows. Three main forgery modes are identified: **fake audio & fake video**, **fake audio & real video**, and **real audio & fake video** [2508.17282]. The dataset also includes **audio-video misalignment**, described as asynchronous temporal forgery, such as manipulated audio with real video or the reverse. In the DDL paper’s terminology, forged audio and video content can be fabricated at **non-aligned/independent time sequences**, and the temporal-domain manipulation repertoire includes **content replacement**, **content deletion**, **content insertion**, and **hybrid / asynchronous forgeries** [2506.23292]. These properties distinguish DDL-AV from benchmarks centered on globally forged clips or tightly synchronized manipulations. A common misconception is to treat audio-visual deepfake datasets as if they test only whether a whole clip is fake. DDL-AV is designed for a harder regime in which manipulation may be partial, modality-specific, or temporally non-overlapping. That design is central to its value as a benchmark for interpretable and forensic-style systems. ## 4. Annotation design and supported tasks The defining technical feature of DDL-AV is its fine-grained temporal annotation scheme. The DDL paper states that, for audio-visual forgeries, the dataset provides **temporal segment labels** specifying the precise timing, including **start/end** of fake segments, for both audio and video streams [2506.23292]. It further notes that audio and video may be forged in **partially overlapping** or **non-overlapping** intervals. The ERF-BA-TFD+ description complements this by stating that DDL-AV provides **frame-level binary labels** for whether a frame or segment is real or fake, as well as **segment-level ground truths** to support both classification and localization [2508.17282]. Taken together, these descriptions place DDL-AV squarely in the category of temporally localized multimodal forensics datasets. The parent DDL initiative is motivated by interpretability. The DDL paper argues that binary classification alone is insufficient in critical domains such as law, and that practical interpretability requires localization evidence [2506.23292]. For DDL-AV, that interpretability is temporal rather than pixel-wise: the benchmark asks not only whether a clip is manipulated, but also when the manipulated content occurs in each modality. ## 5. Evaluation protocol and benchmark usage The reported evaluation on DDL-AV is multi-level and localization-oriented. The ERF-BA-TFD+ paper evaluates both **segment-level** and **full-video** performance and reports **Average Precision** at multiple IoU thresholds—AP@0.5, AP@0.75, and AP@0.95—as well as **Average Recall** at different proposal budgets such as AR@100, AR@50, AR@20, and AR@10 [2508.17282]. It also references **accuracy, precision, recall, and F1 score** as part of the evaluation framework. The metrics are given in the usual forms: $$ \mathrm{AP} = \int_0^1 p(r)\,dr $$ and $$ \mathrm{AR}(K)=\frac{1}{N}\sum_{i=1}^N \max_{j\in S_K}\mathrm{IoU}(g_i,p_j). $$ Reported benchmark results illustrate the dataset’s difficulty. In a baseline comparison, scores after further training dropped from AP@0.5 = 0.9630 and AP@0.75 = 0.8498 on LAV-DF to AP@0.5 = 0.5228 and AP@0.75 = 0.3884 on DDL-AV, with AR@100 = 0.5200 on DDL-AV [2508.17282]. After integrating the UMMA attention mechanism, the same work reports DDL-AV fusion-modality scores of AP@0.5 = 0.9243, AP@0.75 = 0.8050, AP@0.95 = 0.0451, and AR@90 = 0.8246. On a sampled validation set, adding the ERF module increased AP@0.5 from 0.6472 to 0.8214, AP@0.75 from 0.5431 to 0.7287, AR@100 from 0.6513 to 0.7886, and AR@10 from 0.5836 to 0.7397; the competition-set overall score was 0.78. DDL-AV was also used in the **“Workshop on Deepfake Detection, Localization, and Interpretability,” Track 2: Audio-Visual Detection and Localization (DDL-AV)**, where ERF-BA-TFD+ won first place [2508.17282]. This competition context is significant because it operationalizes the dataset not merely as a static corpus but as a challenge benchmark for multimodal detection and temporal localization. ## 6. Research significance, difficulty profile, and limitations The main significance of DDL-AV lies in how it combines multimodal scope with interpretable supervision. The DDL paper positions the broader DDL benchmark as a response to the limitations of prior datasets that provide mostly binary labels, restricted forgery scenarios, limited deepfake diversity, and insufficient scale for complex real-world scenarios [2506.23292]. Within that program, DDL-AV supplies the temporal-localization component for audio-visual content. The difficulty profile reported in downstream work is consistent and specific. The ERF-BA-TFD+ paper highlights **high diversity** in forgery techniques, **long-duration and full-length video analysis**, **audio-video misalignment**, **sparse or subtle manipulations**, and the **need for segment-level localization** as primary challenges when using DDL-AV [2508.17282]. These are not incidental benchmark nuisances; they define the dataset’s research role. DDL-AV is intended to penalize methods that overfit local artifacts, assume synchronous multimodal tampering, or collapse localization into clip-level classification. A plausible implication is that DDL-AV is especially useful for studying robustness under modality discordance and temporal sparsity. Because the benchmark includes manipulated and non-manipulated segments within the same clip, and because audio and video can be forged independently, it creates a setting in which multimodal fusion, temporal proposal generation, and interpretable evidence extraction must all function correctly at once. The official DDL project page is listed at: [2506.23292]. Source: https://www.emergentmind.com/topics/ddl-av-dataset