F5-TTS-DPS: Robust In-the-Wild TTS with EMA and DPS

• The paper presents F5-TTS-DPS, a TTS system that integrates EMA-stabilized supervised fine-tuning to enhance optimization stability and improve UTMOS, WER, and speaker similarity scores.
• It introduces Dual-scoring Prompt Selection (DPS), a two-stage pipeline that filters audio and text prompts using large audio-language and language models to ensure high-quality, aligned conditioning.
• The system builds on the F5-TTS foundation, refining both the data and conditioning layers to achieve robust performance in in-the-wild, noisy environments.

F5-TTS-DPS is a text-to-speech system name used for a WildSpoof 2026 TTS-Track submission built on top of the F5-TTS foundation model for robust speech synthesis from in-the-wild data. Its two defining additions are Exponential Moving Average (EMA) during supervised fine-tuning and Dual-scoring prompt selection (DPS), a two-stage curation pipeline that combines Large Audio-LLM scoring of audio prompts with LLM selection of semantically and prosodically aligned reference text. On the WildSpoof 2026 development set, the system reports UTMOS 3.20, speaker similarity 0.51, WER 8.65%, and a-DCF scores of 0.1582, 0.5233, and 0.2562 across three advanced SASV systems (Sun et al., 22 May 2026).

1. Definition and nomenclature

Within the WildSpoof submission, the acronym “DPS” denotes Dual-scoring prompt selection, not a sampler or a guidance rule. The system name therefore refers to an F5-TTS derivative whose prompt-conditioning pipeline is explicitly curated by dual scoring, and whose fine-tuning is stabilized by EMA (Sun et al., 22 May 2026).

The acronym is not standardized across adjacent F5-TTS literature. The original F5-TTS paper introduces Sway Sampling as its inference-time non-uniform flow-step scheduler and does not use “DPS” as a method name (Chen et al., 2024). The Fast F5-TTS paper on Empirically Pruned Step Sampling states explicitly that the paper does not mention or use “DPS” (Zheng et al., 26 May 2025). In DiTReducio, “DPS” is described as a common abbreviation for Diffusion Posterior Sampling, again not the method used in that work (Huo et al., 11 Sep 2025). Cross-Lingual F5-TTS likewise does not use or define the term “DPS”; there it is only introduced in an explanatory response as shorthand for a duration/pacing system rather than as a paper-defined module (Liu et al., 18 Sep 2025).

This suggests that F5-TTS-DPS is best understood as a submission-specific compound system name rather than a standardized designation for a sampler, solver, or guidance family.

2. Foundation in F5-TTS

F5-TTS is a fully non-autoregressive zero-shot TTS system based on Conditional Flow Matching with a Diffusion Transformer backbone. It inherits E2 TTS’s text-padding idea—characters are padded with a filler token to match mel-spectrogram length—but modifies the architecture with a ConvNeXt V2 text-refinement branch and introduces Sway Sampling at inference. The model uses 100-dimensional log-mel spectrograms at 24 kHz, hop 256, and the base configuration has a 22-layer, 16-head DiT plus a 4-layer ConvNeXt V2 branch, for approximately 336M parameters (Chen et al., 2024).

Its flow-matching formulation is built on the linear path

$\psi_t(x_0) = (1-t)x_0 + t x_1,$

with the OT-CFM objective

$$L_{\mathrm{OT\text{-}CFM}}(\theta) = \mathbb{E}_{t, x_1 \sim q, x_0 \sim p}\left\|v_\theta\!\big((1-t)x_0 + t x_1, t, c\big) - (x_1 - x_0)\right\|^2.$$

At inference, the probability-flow ODE is discretized, typically with Euler updates of the form

$$x_{i+1} = x_i + (t_{i+1} - t_i)\, v_\theta(x_i, t_i, c).$$

Classifier-Free Guidance is used, with the original paper writing

$$v_{t,\mathrm{CFG}} = v_t(\psi_t(x_0), c) + \alpha \big(v_t(\psi_t(x_0), c) - v_t(\psi_t(x_0))\big),$$

and defaulting to $\alpha=2$ (Chen et al., 2024).

F5-TTS-DPS keeps this F5-TTS foundation model as its acoustic generator and adapts the publicly available F5-TTS v1 Base checkpoint pre-trained on Emilia. The WildSpoof report characterizes the base model as a large, non-autoregressive TTS foundation model using flow matching with a diffusion-transformer backbone, with prompt conditioning for voice cloning and supervised fine-tuning that preserves the flow-matching objective (Sun et al., 22 May 2026).

3. EMA-stabilized supervised fine-tuning

A central component of F5-TTS-DPS is EMA-stabilized supervised fine-tuning on noisy, heterogeneous corpora. The reported motivation is specific to in-the-wild speech: environmental noise, reverberation, device variability, transcript imperfections, text–audio misalignment, and variable prompt quality can destabilize optimization and encourage overfitting to artifacts rather than robust conditional generation (Sun et al., 22 May 2026).

The submission maintains a shadow parameter copy updated once per optimizer step:

$$\theta_{\mathrm{EMA}}(t) = \alpha \theta_{\mathrm{EMA}}(t-1) + (1-\alpha)\theta(t),$$

with $\alpha = 0.99$. Gradients are computed on the student weights $\theta$, while evaluation and checkpoint selection are performed using $\theta_{\mathrm{EMA}}$. The report attributes to EMA three functions: smoothing optimization noise, improving generalization and calibration, and producing more stable convergence on heterogeneous corpora (Sun et al., 22 May 2026).

The fine-tuning configuration is explicitly specified. Initialization uses the public F5-TTS v1 Base checkpoint. Training data consists of the TITW-easy and TITW-hard subsets. The learning rate is $1\times 10^{-6}$ with 20,000 warmup steps; gradient accumulation is 1; gradient clipping uses max-norm 1.0; the batch is up to 64 sequences per batch and 38,400 acoustic frames per device; training runs for 10 epochs. The objective remains the original flow-matching loss from F5-TTS, and the report states that no additional adversarial or alignment losses are introduced (Sun et al., 22 May 2026).

The ablation attributed to EMA is quantitatively specific. Relative to supervised fine-tuning without EMA, adding EMA changes UTMOS from 3.06 to 3.18, reduces WER from 10.60% to 9.32%, and increases speaker similarity from 0.489 to 0.492 on the development set (Sun et al., 22 May 2026).

4. Dual-scoring prompt selection

DPS is the submission’s second major addition and serves as a prompt-candidate curation pipeline. Its stated purpose is to improve synthesis fidelity by filtering reference audio and text prompts for quality and alignment. The report emphasizes that prompt quality governs how well the model clones speaker timbre and prosody, particularly under in-the-wild conditions where reference recordings differ sharply in expressiveness, SNR, speaking rate, and segmentation quality (Sun et al., 22 May 2026).

The official system uses a two-stage filter rather than weighted score fusion. In the first stage, a Large Audio-LLM, Qwen2.5-Omni, assigns a score in $$L_{\mathrm{OT\text{-}CFM}}(\theta) = \mathbb{E}_{t, x_1 \sim q, x_0 \sim p}\left\|v_\theta\!\big((1-t)x_0 + t x_1, t, c\big) - (x_1 - x_0)\right\|^2.$$0 for Emotional Richness, Voice Expressiveness, and Prompt Suitability. Candidates that fail an audio-quality threshold $$L_{\mathrm{OT\text{-}CFM}}(\theta) = \mathbb{E}_{t, x_1 \sim q, x_0 \sim p}\left\|v_\theta\!\big((1-t)x_0 + t x_1, t, c\big) - (x_1 - x_0)\right\|^2.$$1 are rejected; the report explicitly notes that the numeric threshold is fixed for the submission but not published. In the second stage, a text-only LLM, Qwen3-30B-A3B, receives the target text together with candidate reference texts and selects the best match according to prosodic alignment, emotional congruence, linguistic compatibility, and overall reference suitability (Sun et al., 22 May 2026).

The pipeline is supplemented by explicit heuristics. Prompts with repeated VAD dropouts, long silences, or music/noise overlays are rejected. Prompts with consistent speaking rate and clear phrase boundaries are preferred. If multiple prompts survive the audio filter, the system prefers those whose transcript length and syntactic structure are closest to the target text. The report also describes accepted cases as clean and expressive prompts with varied intonation and compatible clause structure, and rejected cases as monotone or noisy prompts, prompts with background music, rhetorical-question/declarative mismatches, and mis-segmented references with cross-speaker contamination (Sun et al., 22 May 2026).

The report presents a normalized linear fusion formula only as an optional downstream alternative:

$$L_{\mathrm{OT\text{-}CFM}}(\theta) = \mathbb{E}_{t, x_1 \sim q, x_0 \sim p}\left\|v_\theta\!\big((1-t)x_0 + t x_1, t, c\big) - (x_1 - x_0)\right\|^2.$$2

with acceptance when $$L_{\mathrm{OT\text{-}CFM}}(\theta) = \mathbb{E}_{t, x_1 \sim q, x_0 \sim p}\left\|v_\theta\!\big((1-t)x_0 + t x_1, t, c\big) - (x_1 - x_0)\right\|^2.$$3. The official WildSpoof submission, however, is described as using the two-stage filter rather than combined-score fusion (Sun et al., 22 May 2026).

5. Reported empirical behavior

The development-set results position F5-TTS-DPS as an incremental system over progressively stronger baselines rather than as an isolated configuration. The reported ablation chain begins with baseline F5-TTS at UTMOS 3.06, WER 12.31%, SPK-sim 0.450, and SDS 0.283. Adding supervised fine-tuning changes these values to UTMOS 3.06, WER 10.60%, SPK-sim 0.489, and SDS 0.226. Adding EMA yields UTMOS 3.18, WER 9.32%, SPK-sim 0.492, and SDS 0.181. The full F5-TTS-DPS system reaches UTMOS 3.20, WER 8.65%, SPK-sim 0.508, and SDS 0.108 (Sun et al., 22 May 2026).

The submission also reports DNSMOS 2.61 for the final system and states that evaluation uses the VERSA toolkit for UTMOS, DNSMOS, and WER, ESPnet2-SPK for speaker embeddings, and official SASV-provided a-DCF computations. The development set speaker similarity is described as 0.51 in the summary, while the detailed table gives 0.508 (Sun et al., 22 May 2026).

The headline WildSpoof metric is a-DCF, where lower values indicate that synthesized speech is harder for spoofing-aware speaker verification systems to detect. F5-TTS-DPS reports official scores of 0.1582 for T01, 0.5233 for T02, and 0.2562 for T08, and the report states that these are the best a-DCF scores across all submissions for those three advanced SASV systems. The same source notes that a different “seen speakers” leaderboard protocol reports 0.2798 for team T05 (Sun et al., 22 May 2026).

The ablation narrative attached to these numbers is also explicit. EMA is said to improve naturalness and intelligibility on top of supervised fine-tuning, while DPS is said to deliver further gains in speaker similarity and spoofing difficulty. This suggests that the two additions operate on different failure modes: EMA on optimization stability and generalization, DPS on conditioning quality and prompt–text compatibility (Sun et al., 22 May 2026).

6. Limitations, misconceptions, and position within the F5-TTS ecosystem

The WildSpoof report records several limitations. DPS mis-ranking can admit prompts with subtle style mismatches. EMA adaptation lag is possible because a high decay coefficient can slow adaptation under abrupt distribution shifts. Residual artifacts may remain when prompts have very low SNR or strong music overlays. Generalization beyond the in-the-wild English setup may require retuning DPS thresholds and language-specific front-end normalization (Sun et al., 22 May 2026).

A recurring misconception is to read “DPS” here as Diffusion Posterior Sampling. That interpretation does not match the submission. The original F5-TTS paper uses flow matching, Classifier-Free Guidance, and Sway Sampling, not DPS (Chen et al., 2024). Fast F5-TTS accelerates inference with Empirically Pruned Step Sampling, a training-free non-uniform schedule that achieves 7-step generation with RTF 0.030 on an RTX 3090 while preserving quality, but that work explicitly states that the paper does not mention or use “DPS” (Zheng et al., 26 May 2025). DiTReducio, in turn, reduces per-step module compute through Temporal Skipping and Branch Skipping, and explicitly describes its method as orthogonal to sampler-level methods such as DPM-Solver, DPM-Solver++, and posterior-guided updates, while noting that DPS commonly denotes Diffusion Posterior Sampling (Huo et al., 11 Sep 2025).

This places F5-TTS-DPS in a distinct part of the F5-TTS design space. It is not a new ODE solver, not a branch-selective guidance rule, and not a transcript-free duration system. Its primary contribution is the coupling of EMA-stabilized supervised adaptation with dual-scored prompt curation for noisy, weakly aligned, in-the-wild speech. A plausible implication is that the system is best read as a data-and-conditioning refinement layer on top of the F5-TTS backbone rather than as a modification of the underlying flow-matching dynamics.