---
title: Environmental Sound Deepfake Detection Challenge
url: https://www.emergentmind.com/topics/environmental-sound-deepfake-detection-challenge
type: topic
---

# Environmental Sound Deepfake Detection Challenge

Environmental Sound Deepfake Detection Challenge (ESDD) is a benchmark and competition for deciding whether a 4-second environmental-sound clip is real or fake, motivated by the rapid growth of text-to-audio, audio-to-audio, and later video-to-audio generation systems that can create highly realistic alarms, footsteps, sirens, crowd sounds, and other soundscapes [2508.04529][2512.24140]. Built on the EnvSDD dataset, the challenge is presented as the first large-scale curated benchmark for environmental sound deepfake detection, and it frames the task as binary detection under two deployment-oriented conditions: generalization to unseen generators and black-box low-resource detection [2508.04529]. Unlike speech anti-spoofing, the challenge targets audio with wider acoustic diversity, more varied sound categories, more heterogeneous recording conditions, and more complex spoofing artifacts [2512.15180].

## 1. Origins, scope, and relation to earlier work

Environmental sound deepfake detection emerged later than speech deepfake detection. An important precursor is "Detection of Deepfake Environmental Audio" [2403.17529], which studies the DCASE 2023 Challenge Task 7 Foley sound synthesis setting. That work formulates the task as binary classification, evaluates on 7 sound classes, uses 5,550 nonfake sounds and 25,200 fake sounds, and reports that fake sounds generated by 44 state-of-the-art synthesizers can be detected on average with 98.02% accuracy by an MS-CLAP plus MLP detector [2403.17529]. The same paper also reports that CLAP embeddings provide about a 10% improvement over VGGish and about a 5% improvement over each PANN variant [2403.17529].

The ESDD challenge departs from that earlier Foley-centered setting in two ways. First, EnvSDD is substantially larger and explicitly structured for cross-generator evaluation rather than only average in-distribution detection performance [2508.04529][2505.19203]. Second, the challenge treats environmental audio as a domain with monophonic and polyphonic content, broad scene and event diversity, and weaker linguistic structure than speech, making direct transfer from speech anti-spoofing unreliable [2505.19203][2603.04865]. The benchmark therefore operationalizes a shift from proof-of-concept environmental deepfake detection to robustness-oriented evaluation.

## 2. EnvSDD benchmark design

EnvSDD is described as the first large-scale curated dataset for environmental sound deepfake detection. The dataset contains 45.25 hours of real audio and 316.7 hours of fake audio, and the challenge is organized around two tracks: Track 1, generalization to unseen TTA and ATA generators, and Track 2, very low-resource black-box spoofing detection [2508.04529][2512.15180]. For Track 1, all audio is resampled to 16 kHz and split into 4-second clips [2508.04529].

The real subset is assembled from open-source environmental sound datasets including UrbanSound8K, TAU UAS 2019 Open Dev, TUT SED 2016, TUT SED 2017 Dev, TUT SED 2017 Eval, DCASE 2023 Task 7 Dev, and Clotho [2508.04529]. The fake subset is generated by seven systems: G01 AudioLDM, G02 AudioLDM 2, G03 AudioGen, G04 AudioLDM (ATA), G05 AudioLCM, G06 TangoFlux, and G07 AudioLDM 2 (ATA) [2508.04529].

| Split | Track 1 | Track 2 |
|---|---:|---:|
| Training | 27,811 real / 111,244 fake | 270 real / 1,083 fake |
| Validation | 7,942 real / 31,768 fake | 90 real / 361 fake |
| Evaluation | 478 real / 1,522 fake | 973 real / 4,014 fake |
| Test | 1,000 real / 3,000 fake | 1,994 real / 7,980 fake |

Track 1 is defined so that training and validation use G01-G04, while evaluation and test use unseen generators G05-G07 [2508.04529][2512.24140]. Track 2 is defined as black-box and low-resource; one evaluation-plan description states that only 1% of the total development data is black-box data [2508.04529]. After the challenge, the Track 2 overview identifies the undisclosed test-time black-box generators as the video-to-audio systems FoleyCrafter and Diff-Foley, with real video samples drawn from VGGSound [2512.24140][2603.04865].

## 3. Evaluation protocol and official baselines

The official metric is Equal Error Rate (EER), with lower values indicating better balanced detection performance. Operationally, EER is the threshold where

$$\mathrm{FAR} = \mathrm{FRR}.$$

Participants submit a TXT score file containing the clip ID and a confidence score for whether the clip is bona fide [2508.04529]. The challenge uses an evaluation set for progress ranking and a test set for final ranking [2512.15180][2508.04529].

Two baselines are provided: AASIST and BEATs+AASIST. AASIST is an end-to-end anti-spoofing system with 0.30M parameters, while BEATs+AASIST uses BEATs as a front-end encoder trained on AudioSet-2M and has 90.73M parameters [2508.04529]. Their reported performance illustrates the central difficulty of generalization. On Track 1, AASIST obtains validation, evaluation, and test EERs of 0.92%, 15.26%, and 15.02%, while BEATs+AASIST obtains 0.10%, 14.21%, and 13.20% [2508.04529]. On Track 2, AASIST obtains 1.11%, 15.72%, and 15.40%, while BEATs+AASIST obtains 0.16%, 12.64%, and 12.48% [2508.04529]. The evaluation plan explicitly notes that the low validation EERs contrasted with much higher evaluation and test EERs demonstrate severe generalization issues [2508.04529].

For both baselines, the published training protocol uses batch size 16, Adam, weight decay 0.0001, learning rate 0.001 for AASIST, learning rate 0.00001 for BEATs+AASIST fine-tuning, maximum 50 epochs, and early stopping if validation loss does not improve for 5 consecutive epochs [2508.04529].

The challenge was recognized as one of the ICASSP 2026 Grand Challenges. The overview reports 97 registered teams, 1,359 valid submissions for Track 1, and 389 valid submissions for Track 2 [2512.24140].

## 4. Dominant methodological patterns

The challenge overview reports that most top teams used strong pretrained acoustic encoders such as BEATs, EAT, and SSLAM, typically paired with more expressive back-ends or fusion modules rather than relying on a single end-to-end model [2512.24140]. Common back-end designs included AASIST-style anti-spoofing classifiers, bidirectional selective state-space models, lightweight feed-forward heads, top-$k$ layer fusion, and multi-branch architectures [2512.24140][2512.20369]. The overview also reports that ensembling was pervasive and that single-model systems were generally weaker than fused systems [2512.24140].

Augmentation and regularization were similarly prominent. The overview names vocoder-generated fake audio augmentation, compression simulation, loudness normalization, semantically aligned data construction, Cut-and-Mix augmentation, LoRA fine-tuning, domain adversarial training, and ArcFace loss among successful strategies [2512.24140]. A separate Track 1 system paper from BUT reports that general audio self-supervised models such as BEATs, EAT, and Dasheng outperform speech-specific SSLs for environmental sound deepfake detection, attributing the difference to broader non-speech acoustic pretraining [2512.08319].

A closely related pattern is intermediate-layer exploitation. EnvSSLAM-FFN freezes an SSLAM encoder and fuses layers 4-9 with learnable softmax weights, reporting that middle-layer representations are more informative for anti-spoofing than shallow or deep layers [2512.20369]. BEAT2AASIST similarly argues that relying only on the final transformer layer is suboptimal and instead fuses top-$k$ BEATs layers using concatenation, CNN-gated fusion, or SE-gated fusion [2512.15180]. Across systems, the common architectural thesis is that environmental spoofing cues are distributed across representation depth, frequency region, latent channel, and temporal context.

## 5. Representative systems and leaderboard results

Several published challenge submissions illustrate different technical responses to the two-track design.

| System | Main mechanism | Reported outcome |
|---|---|---|
| BEAT2AASIST | Frequency/channel splitting, dual AASIST branches, top-$k$ layer fusion, vocoder augmentation | 1.60% Track 1 test EER ensemble; 0.35% Track 2 test EER ensemble; third-place Track 2 [2512.15180] |
| EnvSSLAM-FFN | Frozen SSLAM, layer fusion from layers 4-9, lightweight FFN, class-weighted BCE | 1.20% Track 1 test EER; 1.05% Track 2 test EER [2512.20369] |
| ATCA / ATCA-ens | Audio captioning, audio-text cross-attention, stacked GRU, stacked regression ensemble | 11.22% Track 1 test EER; 11.98% Track 2 test EER [2512.06041] |
| BUT fusion system | Diverse SSL front-ends, MHFA back-end, DSU feature-domain augmentation | 4.38% Track 1 final evaluation EER [2512.08319] |

BEAT2AASIST extends the official BEATs-AASIST baseline by splitting BEATs-derived representations either along the frequency dimension or along the channel dimension, processing the two halves with independent AASIST branches, optionally fusing multiple top transformer layers, and using vocoder-based augmentation with HiFi-GAN, BigV-GAN, and UnivNet [2512.15180]. Its published results suggest different operating regimes for the two tracks: channel splitting with SE-gated fusion improves Track 1, while frequency splitting with CNN-gated fusion improves Track 2 [2512.15180]. The same paper notes that changing from $k=4$ to $k=10$ does not always help and that ensemble performance is best in both tracks [2512.15180].

EnvSSLAM-FFN adopts a different strategy. It keeps the SSL front-end frozen, uses attentive statistics pooling and a lightweight feed-forward back-end, and trains with a class-weighted binary cross-entropy objective that gives higher weight to bona-fide samples according to the inverse class-frequency ratio [2512.20369]. The reported Track 1 and Track 2 test EERs of 1.20% and 1.05% are far below the official baselines [2512.20369].

Nomi’s ATCA system introduces an audio-text cross-attention model built from an audio branch, a text branch using captions generated in AudioSet, AudioCaps, and Clotho styles, a cross-attention module where audio queries text, and a stacked GRU [2512.06041]. On Track 1, ATCA improves on the BEATs+AASIST baseline from 13.20% to 11.28% test EER, and its ensemble reaches 11.22%; on Track 2, the single ATCA model is slightly worse than the baseline at 12.64% versus 12.48%, while the ensemble reaches 11.98% [2512.06041]. The authors attribute the weaker Track 2 behavior to reduced caption quality or reduced semantic reliability under low-resource black-box conditions [2512.06041].

At the challenge level, the strongest systems outperformed the published representative submissions above. The overview reports that Track 1 test EER reached 0.30% for AHU with an EAT + AASIST ensemble of 5 models, followed by DFKI at 0.80% and CUC at 1.20%; on Track 2, DFKI and AHU both reached 0.25%, CAU reached 0.35%, and CUC reached 1.05% [2512.24140]. The same overview notes that Track 2 was not harder than expected, since top systems generally obtained lower EERs on Track 2 than on Track 1 [2512.24140].

## 6. Core findings, limitations, and subsequent developments

The first ESDD challenge established several empirical regularities. The most important is that in-domain fitting is relatively easy, while unseen-generator robustness is the decisive criterion [2603.04865][2508.04529]. The challenge analysis identifies TangoFlux (G06) as the hardest unseen generator in Track 1: the baselines obtain 19.00% EER for BEATs+AASIST and 20.10% EER for AASIST on G06, while top systems remain much more robust [2603.04865]. This indicates that high-fidelity generators with fewer obvious artifacts are the primary threat model for environmental audio forensics.

The challenge also clarified that environmental sound deepfake detection cannot be reduced to speech anti-spoofing transferred unchanged. EnvSDD benchmarking already showed that a speech-pretrained W2V2+AASIST front-end can perform well in-domain yet collapse under unseen environmental conditions, whereas BEATs+AASIST is more robust across unseen generators and unseen datasets [2505.19203]. A later large-scale sound-effects benchmark, SynSFX, strengthens the same conclusion: in a strict zero-shot protocol on synthetic sound effects, AASIST yields 60.84% EER, RawNet2 yields 49.94% EER, and EAT-AASIST yields 23.71% EER, showing that speech-centric detectors do not transfer well to synthetic environmental audio and that unseen synthetic generators remain the main bottleneck [2607.04848].

The legacy of the challenge is visible in successor benchmarks. ESDD2 moves from clip-level binary detection to five-class component-level spoofing detection in mixed speech-environment recordings, and its baseline reports a test Macro-F1 of 0.6327 with an environmental EER of 0.4279 versus a speech EER of 0.1978, indicating that the spoofed environmental component remains especially difficult to detect [2606.10791]. AT-ADD Track 2 further extends the problem to unknown audio type, requiring type-agnostic real-versus-fake detection across speech, sound, singing, and music [2604.08184].

A plausible implication is that the first ESDD challenge redefined environmental audio forensics around three technical priorities: generator-invariant representation learning, robustness under distribution shift, and evaluation beyond speech-specific assumptions. The published systems and follow-on benchmarks suggest that these priorities remain unresolved rather than exhausted.

Source: https://www.emergentmind.com/topics/environmental-sound-deepfake-detection-challenge