DMOSpeech 2: Metric-Optimized Zero-Shot TTS

• The paper demonstrates that applying reinforcement learning for duration prediction significantly improves intelligibility and speaker similarity in TTS.
• It introduces teacher-guided sampling to recover prosodic diversity while maintaining efficiency in a flow-matching diffusion pipeline.
• The system integrates metric optimization with targeted RL to refine a discrete duration predictor, addressing a previously unoptimized component in TTS.

DMOSpeech 2 is a zero-shot text-to-speech system that extends direct metric optimization from the acoustic generator to duration prediction within a flow-matching diffusion TTS pipeline. It builds on the teacher–student framework introduced by DMOSpeech, but addresses a previously unoptimized component—the duration predictor—through reinforcement learning, specifically group relative preference optimization (GRPO) with speaker similarity and word error rate as reward signals. The system also introduces teacher-guided sampling, a hybrid teacher–student inference procedure intended to recover prosodic diversity while preserving the efficiency of distilled generation. In the reported evaluations, DMOSpeech 2 forms a more complete metric-optimized synthesis pipeline and reduces sampling steps by half without quality degradation (Li et al., 20 Jul 2025).

1. Historical placement and motivation

DMOSpeech 2 is situated in the line of zero-shot diffusion and flow-matching TTS systems that generate mel-spectrograms from noise conditioned on text and a speech prompt. Its immediate predecessor, DMOSpeech, established a teacher diffusion model, a student model distilled with distribution matching distillation, and direct metric optimization of the generator through a CTC-based ASR model and a speaker verification model, thereby enabling optimization for intelligibility and speaker similarity within a fast student generator (Li et al., 2024).

The central motivation for DMOSpeech 2 is that duration prediction remained outside that optimization loop. In flow-matching diffusion TTS, the full utterance duration must be known before denoising because the generator inpaints a fixed-length mel canvas. DMOSpeech 2 identifies this as a bottleneck because duration directly affects alignment and intelligibility, prosodic naturalness, and speaker similarity. Appendix A.1 in the paper isolates this effect on Seed-TTS-en with the same generator but different duration sources: ground truth audio gives SIM 0.734 and WER 2.143, ground truth duration gives SIM 0.697 and WER 1.821, a speaking rate heuristic gives SIM 0.682 and WER 2.028, a supervised duration predictor without reinforcement learning gives SIM 0.672 and WER 3.750, best-of-8 duration sampling gives SIM 0.724 and WER 1.723, and the DMOSpeech 2 duration model gives SIM 0.698 and WER 1.752 (Li et al., 20 Jul 2025).

This framing places DMOSpeech 2 in a specific methodological niche. Rather than applying reinforcement learning to the entire generative stack, it applies targeted reinforcement learning to a discrete, non-differentiable control variable whose errors propagate strongly into downstream synthesis quality. A plausible implication is that the work treats duration as the remaining high-leverage component in a pipeline already optimized for WER and speaker similarity at the acoustic level.

2. System architecture and metric-optimized pipeline

At the system level, DMOSpeech 2 consists of a text encoder, a duration predictor or speech length predictor $\mathcal{P}_\phi$, a flow-matching acoustic model with teacher $G_\Theta$ and student $G_\theta$, and a Vocos vocoder. The teacher is a full F5-TTS flow model trained on Emilia, approximately 95k hours of multilingual in-the-wild speech, with standard flow-matching loss and 32 denoising steps. The student is a distilled 4-step flow model optimized with Improved DMD (DMD2), adversarial training with a Conformer discriminator, direct WER optimization via CTC ASR, and direct speaker similarity optimization via speaker verification. The duration predictor is first trained supervised by cross-entropy on ground-truth remaining lengths and then fine-tuned by GRPO. Teacher-guided sampling can be enabled at inference, with early flow steps computed by the teacher and later steps by the student (Li et al., 20 Jul 2025).

The architecture thereby separates two forms of optimization. The acoustic generator remains differentiably optimized through metric surrogates, while the duration predictor is optimized through a stochastic policy objective. This division reflects a structural property of the task: generator outputs admit gradient-based supervision through ASR and SV modules, but discrete duration decisions do not.

The duration predictor itself is an encoder–decoder Transformer similar to DiTTo-TTS but repurposed to predict remaining length. It uses a 4-layer Transformer text encoder, a 4-layer Transformer audio decoder that is causal along time and cross-attends to text, 8 attention heads, hidden dimension 512, and a final linear plus softmax over 300 length classes. The maximum total duration is approximately 30 seconds, discretized into 300 bins of 100 ms each. Its supervised target for an utterance of length $L_i$ frames is the remaining length $L_i - \tau$ at frame index $\tau$ (Li et al., 20 Jul 2025).

Within this design, DMOSpeech 2 can be understood as a pipeline in which both the acoustic realization and the temporal scaffold are aligned to the same evaluation-oriented objectives. This is the principal architectural distinction from the original DMOSpeech, where only the generator was directly metric-optimized (Li et al., 2024).

3. Reinforcement learning formulation for duration prediction

DMOSpeech 2 formulates duration prediction as a stochastic policy over discrete utterance lengths,

$$\pi_\phi(L \mid \mathbf{x}, \mathbf{p}) = \mathcal{P}_\phi(\mathbf{x}, \mathbf{p}),$$

where the state is the text $\mathbf{x}$ together with the speech prompt $\mathbf{p}$, and the action is the total or remaining length $L$ in frames. The environment consists of the 4-step student generator and downstream reward models: given a sampled duration, the system samples noise of matching shape, generates a mel spectrogram, synthesizes speech, and evaluates the result with ASR and speaker verification (Li et al., 20 Jul 2025).

For each input pair $G_\Theta$0, the policy is temperature-scaled with Gumbel-softmax at $G_\Theta$1, and a group of $G_\Theta$2 candidate durations is sampled. For each candidate duration $G_\Theta$3, the student generates speech $G_\Theta$4 with $G_\Theta$5. The scalar reward is

$G_\Theta$6

with $G_\Theta$7, where $G_\Theta$8 is a CTC ASR model on mel-spectrograms and $G_\Theta$9 is a speaker verification encoder producing embeddings $G_\theta$0 and $G_\theta$1 (Li et al., 20 Jul 2025).

The group-relative component is defined by normalizing rewards within the sampled set for the same input:

$G_\theta$2

$G_\theta$3

Batches are skipped when the reward spread is too small, specifically when $G_\theta$4. The update uses a current policy $G_\theta$5, an old policy $G_\theta$6, and a frozen reference policy $G_\theta$7 derived from the supervised duration model. With ratio

$G_\theta$8

the per-sample objective is

$G_\theta$9

with $L_i$0 and $L_i$1, where the KL term is $L_i$2. The training objective minimizes the negative mean over the group and data distribution (Li et al., 20 Jul 2025).

The paper characterizes this as bandit-style, single-step episodic reinforcement learning rather than temporally extended control. That distinction is consequential: there is no frame-level credit assignment. Instead, GRPO exploits relative preferences among multiple plausible durations for the same utterance. This suggests that the method is tailored to a regime where the action space is modest—300 bins—and where several length hypotheses may all be locally plausible but measurably different under ASR and speaker-similarity rewards.

4. Teacher-guided sampling and diversity recovery

A second innovation in DMOSpeech 2 is teacher-guided sampling, introduced to address mode shrinkage in aggressively distilled students. The paper reports that DMD-distilled students with very few steps tend to preserve audio quality while under-representing rare prosodic patterns. The evidence is that UTMOS distributions are similar between teacher and student, but F0 distributions are substantially narrower for the student. Over 50 samples per text/prompt pair, the coefficient of variation of F0 is $L_i$3 for the teacher, $L_i$4 for the student, and $L_i$5 for the teacher-guided hybrid (Li et al., 20 Jul 2025).

The proposed remedy is a hybrid sampling schedule in which early high-noise steps are performed by the teacher and later low-noise steps by the student. The paper’s interpretation is structural: early steps set global prosodic structure, including durations, pauses, intonation contours, and alignment, whereas later steps refine spectral and fine detail. In teacher-guided sampling, noise $L_i$6 is first integrated through the teacher up to a switching time $L_i$7, then the resulting state is passed to the student for the final steps. In the reported main configuration, this corresponds to teacher 14 steps plus student 2 steps, for 16 total steps (Li et al., 20 Jul 2025).

Empirically, this hybrid restores much of the teacher’s prosodic diversity while preserving the student’s metric-optimized behavior. On Seed-TTS-en, the teacher-guided hybrid reports WER 1.738, SIM 0.699, and RTF 0.0941, compared with WER 1.752, SIM 0.698, and RTF 0.0316 for the 4-step student and WER 1.947, SIM 0.662, and RTF 0.1671 for the 32-step teacher. On Seed-TTS-zh, the hybrid gives CER 1.468 and SIM 0.760, compared with CER 1.527 and SIM 0.760 for the student and CER 1.695 and SIM 0.750 for the teacher (Li et al., 20 Jul 2025).

The significance of teacher-guided sampling is therefore not only computational. It also functions as a mechanism for decoupling two roles of the teacher: preserving early-stage structural uncertainty while allowing late-stage refinement to remain under the student’s metric-oriented control. This suggests a hybrid view of distillation in TTS, where the teacher is not merely a training signal but an inference-time source of controlled diversity.

5. Training regime and empirical performance

DMOSpeech 2 is trained on the Emilia corpus, described as approximately 95k hours of multilingual, in-the-wild speech. Evaluation is conducted on Seed-TTS test-en, comprising 1088 samples from CommonVoice, and Seed-TTS test-zh, comprising 2020 samples from DiDiSpeech, in a cross-sentence zero-shot TTS setting with a short prompt, its transcription, and a different target text (Li et al., 20 Jul 2025).

The teacher is a 0.3B parameter F5-TTS model trained for 2M steps on Emilia with AdamW, peak learning rate $L_i$8, 20k warmup steps, and linear decay. The student has the same base scale and is distilled with DMD2 for an additional 200k steps, then optimized with flow-matching loss, multi-step DMD sampling, a Conformer-based discriminator, a CTC loss, and an SV loss. The metric-loss schedule sets $L_i$9 for the first 5000 iterations and then 1, and $L_i - \tau$0 for the first 10k iterations and then 1, with adversarial weight $L_i - \tau$1. The duration predictor is trained supervised for 85k steps with AdamW at learning rate $L_i - \tau$2, then fine-tuned with GRPO for 1500 steps. All experiments use 8 NVIDIA H100 GPUs (Li et al., 20 Jul 2025).

The main internal comparison shows the effect of duration reinforcement learning. On Seed-TTS-en, the 4-step student with RL duration achieves WER 1.752 and SIM 0.698, whereas the same student without duration RL gives WER 3.750 and SIM 0.672. On Seed-TTS-zh, the RL duration model gives CER 1.527 and SIM 0.760, compared with CER 2.000 and SIM 0.750 without RL. Subjective pairwise comparisons against the student without duration RL show CMOS-N = $L_i - \tau$3 and CMOS-S = $L_i - \tau$4 in English, and CMOS-N = $L_i - \tau$5 and CMOS-S = $L_i - \tau$6 in Chinese, indicating that the RL-duration variant is preferred (Li et al., 20 Jul 2025).

The broader system-level comparison places DMOSpeech 2 against substantially larger or slower systems. The 0.3B, 4-step student reports WER 1.752 and SIM 0.698 on English, CER 1.527 and SIM 0.760 on Chinese, with RTF 0.032. The teacher F5-TTS baseline reports WER 1.947, SIM 0.662, CER 1.695, SIM 0.750, and RTF 0.167. Other reported baselines include CosyVoice 2 at WER 3.358, SIM 0.641, CER 1.582, SIM 0.754, and RTF 0.527; Spark-TTS at WER 2.308, SIM 0.572, CER 1.717, SIM 0.657, and RTF 1.784; MaskGCT at WER 2.622, SIM 0.713, CER 2.395, SIM 0.772, and RTF 2.397; and LLaSA-8B at WER 3.994, SIM 0.594, CER 4.214, SIM 0.671, and RTF 1.374. Within these comparisons, DMOSpeech 2 is reported as best or second-best in WER/CER while being the fastest system in the table (Li et al., 20 Jul 2025).

These results define the empirical identity of DMOSpeech 2. It is not presented merely as a quality-improved successor to DMOSpeech, but as a system in which targeted RL on duration closes a measurable optimization gap that persists even after direct metric optimization of the generator itself.

6. Limitations, scope, and research significance

The paper identifies several limitations. First, duration RL depends on surrogate metric models: the WER proxy is a specific ASR system and the similarity reward is a specific speaker encoder, so the learned policy is aligned to those models rather than directly to human judgments. Second, reinforcement learning remains computationally expensive because each policy update requires full speech generation for multiple candidate durations, with group size $L_i - \tau$7. Third, evaluation is limited to English and Chinese, although training uses multilingual Emilia data. Fourth, teacher-guided sampling requires loading both teacher and student, increasing the parameter footprint and reducing efficiency relative to the pure 4-step student. Fifth, the RL scope is restricted to duration, while the generator itself continues to be optimized through differentiable metric losses rather than reinforcement learning (Li et al., 20 Jul 2025).

The paper also outlines future directions: targeted reinforcement learning for other discrete or non-differentiable components such as prosody control, pause insertion, or the teacher sampling schedule; additional reward components such as MOS predictors, emotion and style metrics, or listener preference models; extension to more languages and domains; and improvements in the robustness of the ASR and speaker-verification reward models (Li et al., 20 Jul 2025).

In the context of the DMOSpeech line, the main contribution of DMOSpeech 2 is conceptual as much as empirical. DMOSpeech showed that a distilled diffusion-style TTS generator could be directly optimized for intelligibility and speaker similarity through differentiable surrogates (Li et al., 2024). DMOSpeech 2 extends that principle to a component that is intrinsically discrete and externally coupled to the generator. The resulting system exemplifies a broader design pattern: use differentiable direct metric optimization where gradient pathways are available, and use compact RL where task-critical decisions remain non-differentiable. This suggests a modular route toward increasingly metric-aligned speech synthesis systems, in which timing, prosody, and other control variables are optimized against the same downstream criteria as the acoustic model itself.