VoiceTTA: RL-Based Test-Time Adaptation for TTS

• VoiceTTA is a reinforcement learning-based test-time adaptation method that optimizes learnable prefixes to improve zero-shot TTS voice imitation.
• It adapts to uncommon speech prompts like dialects, children's voices, slurred speech, and crosstalk while preserving intelligibility and naturalness.
• Built on F5-TTS, VoiceTTA leverages style, speaker similarity, and intelligibility rewards to refine synthesis quality with minimal parameter adjustments.

VoiceTTA is a reinforcement learning-based test-time adaptation method for zero-shot text-to-speech that improves voice imitation of pretrained zero-shot TTS models by optimizing learnable prefixes at inference time rather than fine-tuning the backbone. Implemented on top of the flow matching-based F5-TTS, it targets uncommon speech prompts such as crosstalk, dialects, children’s voices, and slurred speech, and combines style rewards based on coefficient-of-variation differences of $F0$ and energy with speaker similarity and intelligibility rewards computed from pretrained models (Xie et al., 25 Jun 2026).

1. Problem formulation and scope

VoiceTTA is motivated by two limitations of contemporary zero-shot TTS. First, zero-shot systems trained on common domains such as audiobooks, podcasts, and TV shows often preserve timbre while failing to imitate out-of-domain prosodic behavior, including strong regional dialects, children’s voices, slurred speech, and sketch-style crosstalk. Second, conventional adaptation by fine-tuning requires large, high-quality datasets and does not scale to rapid personalization. VoiceTTA addresses both issues by treating adaptation as a test-time reinforcement learning problem over a very small parameter set, leaving the pretrained backbone frozen (Xie et al., 25 Jun 2026).

The method is explicitly framed as test-time adaptation. For a new reference prompt and target text, VoiceTTA performs online optimization during inference, using only the prompt itself and auxiliary rewards that can be computed from synthesized outputs. This makes the method distinct from corpus-level speaker fine-tuning and from purely embedding-based zero-shot conditioning. Its central claim is not that the backbone architecture changes, but that the imitation behavior of an existing zero-shot model can be materially improved at inference time through reward-driven prefix optimization.

2. Backbone model and adaptation parameters

VoiceTTA is built on F5-TTS, a flow matching-based TTS model that maps a latent variable sampled from a Gaussian prior to a mel-spectrogram conditioned on text and a speech prompt. Within this backbone, the only learnable test-time parameters are four learnable prefix embeddings prepended to the input of the first DiT layer. The total adaptation footprint is approximately $16$ KB per unseen speaker, and all backbone parameters remain frozen throughout adaptation (Xie et al., 25 Jun 2026).

This design makes VoiceTTA parameter-efficient in a stricter sense than conventional speaker fine-tuning. The prefixes function as a compact control interface into the pretrained flow model, analogous in spirit to prompt tuning, but localized to inference-time optimization rather than supervised pretraining. The paper emphasizes that these prefixes are introduced only at TTA time. After adaptation, the optimized prefixes can be reused for future utterances for the same speaker or style, while unrelated test cases reinitialize the prefixes to prevent adaptation leakage across samples.

The flow matching backbone is trained to learn a time-dependent velocity field. VoiceTTA inherits the standard flow matching loss

$$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$

where $v_\theta(x_t,t)$ is the model-predicted velocity field and $u_t(x_t\mid o)$ is the target velocity field transporting the prior toward a target mel-spectrogram $o$ (Xie et al., 25 Jun 2026).

3. Reinforcement learning formulation

VoiceTTA formulates adaptation as RL over the learnable prefixes. The reference audio $x$, target text $p$, and current prefix values define the implicit state; the action is the generation of a candidate utterance $o_i$; and the policy is parameterized by the prefixes $\theta$. Optimization uses Group Relative Preference Optimization (GRPO), with groups of synthesized candidates scored by style, speaker similarity, and intelligibility rewards (Xie et al., 25 Jun 2026).

For a group of $16$0 candidates, VoiceTTA computes a normalized advantage

$16$1

The GRPO objective is

$16$2

The paper omits a KL penalty term because only a tiny set of prefixes is optimized. It also treats $16$3 as a log-probability proxy derived from the flow matching objective rather than from an explicit token-level likelihood.

That proxy is defined by assuming

$16$4

Under this approximation, the policy comparison becomes

$16$5

This is a notable aspect of the method: RL is executed on top of a flow-matching generator without introducing a separate discrete likelihood model. A plausible implication is that VoiceTTA is tied less to a particular decoder parameterization than to the availability of a tractable per-sample generative loss that can serve as a probability surrogate.

4. Reward design and adaptation procedure

VoiceTTA uses four rewards. Two are style rewards based on coefficient-of-variation differences of $16$6 and energy. For a sequence $16$7, the coefficient of variation is

$16$8

Let $16$9 and $$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$0 denote generated and reference $$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$1 contours over voiced frames. The $$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$2-CV reward is

$$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$3

Analogously, using frame-wise energy computed as the sum of mel filter-bank magnitudes per frame, the Energy-CV reward is

$$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$4

These rewards favor matching relative variation patterns rather than absolute pitch or loudness (Xie et al., 25 Jun 2026).

The third reward is speaker similarity: $$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$5 where $$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$6 is a pretrained speaker encoder and $$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$7, $$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$8 are generated and reference waveforms. The fourth reward is intelligibility, computed from Whisper-Large V3 word error rate between the synthesized utterance and the target text. Each reward type is normalized across the $$\mathcal{L}_{\text{FM},\theta}(o) = \mathbb{E}_{t, p(x_t \mid o)} \left[ \left| v_\theta(x_t, t) - u_t(x_t\mid o) \right|^2 \right],$$9 candidates to $v_\theta(x_t,t)$0, producing $v_\theta(x_t,t)$1, $v_\theta(x_t,t)$2, $v_\theta(x_t,t)$3, and $v_\theta(x_t,t)$4. The combined reward is

$v_\theta(x_t,t)$5

with $v_\theta(x_t,t)$6, $v_\theta(x_t,t)$7, and $v_\theta(x_t,t)$8 (Xie et al., 25 Jun 2026).

At test time, VoiceTTA samples $v_\theta(x_t,t)$9 candidates per GRPO step, drawing temperatures from $u_t(x_t\mid o)$0. It performs $u_t(x_t\mid o)$1 GRPO steps with learning rate $u_t(x_t\mid o)$2 and $u_t(x_t\mid o)$3 warmup on an NVIDIA RTX 6000 Ada GPU. Since each step synthesizes four candidates, a full adaptation run entails $u_t(x_t\mid o)$4 candidate generations for one speaker or style prompt. This makes adaptation materially more expensive than plain zero-shot inference, but the cost is amortized if the adapted prefixes are reused for multiple utterances.

5. Empirical results and ablation behavior

VoiceTTA is evaluated on an internal uncommon-style dataset containing $u_t(x_t\mid o)$5 speech prompts—$u_t(x_t\mid o)$6 accented, $u_t(x_t\mid o)$7 children’s voices, $u_t(x_t\mid o)$8 slurred speech, and $u_t(x_t\mid o)$9 Chinese sketches—and on a KeSpeech dialect subset with $o$0 utterances from $o$1 Chinese dialects, $o$2 per dialect. Across these five scenarios, VoiceTTA achieves the best average WER and speaker similarity among the reported systems: WER $o$3 versus $o$4 for F5-TTS, $o$5 for MaskGCT, $o$6 for CosyVoice, and $o$7 for Vevo; and S-SIM $o$8 versus $o$9 for F5-TTS, $x$0 for MaskGCT, $x$1 for CosyVoice, and $x$2 for Vevo (Xie et al., 25 Jun 2026).

Subjective evaluation follows the same pattern. Average style similarity S-MOS is $x$3, exceeding CosyVoice at $x$4, MaskGCT at $x$5, Vevo at $x$6, and F5-TTS at $x$7. Naturalness N-MOS is $x$8, essentially matching F5-TTS at $x$9, below CosyVoice at $p$0, and above MaskGCT at $p$1 and Vevo at $p$2. The empirical pattern is therefore not a pure naturalness gain; rather, VoiceTTA principally increases style similarity while preserving the backbone’s intelligibility and naturalness envelope.

Per-scenario improvements are especially pronounced in harder stylistic conditions. On slurred speech, WER improves from $p$3 to $p$4, S-SIM from $p$5 to $p$6, and S-MOS from $p$7 to $p$8. On Chinese dialects, WER improves from $p$9 to $o_i$0, S-SIM from $o_i$1 to $o_i$2, and S-MOS from $o_i$3 to $o_i$4. On accented speech, WER remains essentially unchanged at $o_i$5 versus $o_i$6, but S-SIM rises from $o_i$7 to $o_i$8 and S-MOS from $o_i$9 to $\theta$0 (Xie et al., 25 Jun 2026).

Ablation results clarify the role of each reward term. Optimizing only the intelligibility reward yields the best WER, $\theta$1, but collapses S-SIM to $\theta$2. Using only style rewards based on $\theta$3, $\theta$4, and $\theta$5 improves S-SIM to $\theta$6 but degrades WER to $\theta$7. The full reward restores the trade-off, with WER $\theta$8 and S-SIM $\theta$9. The number of prefixes also matters: more prefixes improve S-SIM on adapted-domain evaluations, but too many degrade performance on Seed-TTS test-en, and the paper selects four prefixes as the operating compromise. Temperature is similarly constrained; large $16$00 values sharply reduce intelligibility, which motivates the restricted range $16$01.

6. Position within TTS research and common confusions

VoiceTTA belongs to a broader class of parameter-efficient or inference-time adaptation methods, but its specific mechanism is unusual. VoiceTailor adapts a diffusion-based TTS system by inserting LoRA into attention modules of a pretrained diffusion decoder and fine-tuning only $16$02 of parameters, whereas VoiceTTA keeps the backbone fully frozen and optimizes only four learnable prefixes via GRPO at inference time (Kim et al., 2024). TTS-1 and TTS-1-Max pursue a different route again: they rely on in-context learning from prompt audio tokens and RL alignment during training, not speaker-specific prefix optimization at test time (Atamanenko et al., 22 Jul 2025).

VoiceTTA is also orthogonal to recent architectural innovations in speech generation. TADA proposes synchronous one-to-one text-acoustic alignment with a flow matching head inside a single-stream LLM, while VoxCPM uses tokenizer-free hierarchical semantic-acoustic modeling with an internal differentiable quantization bottleneck and a local diffusion decoder (Dang et al., 26 Feb 2026, Zhou et al., 29 Sep 2025). This suggests that VoiceTTA’s core idea—a reward-driven, parameter-light adaptation layer—could in principle be paired with different backbone representations, though that extrapolation is not demonstrated in the paper.

The name is potentially confusing because other recent works use closely related acronyms for unrelated problems. “Ada-TTA” addresses low-resource text-to-talking avatar synthesis with zero-shot multi-speaker TTS and a NeRF-based talking-face renderer (Ye et al., 2023). “TTA: Transcribe, Translate and Alignment for Cross-lingual Speech Representation” is a multilingual speech encoder for ASR, speech translation, and speech-text alignment, intended for speech-LLM integration (Liu et al., 18 Nov 2025). VoiceTTA, by contrast, is specifically a reinforcement learning-based test-time adaptation method for zero-shot TTS.

Its limitations follow directly from its design. The adaptation phase is substantially heavier than standard inference because it requires $16$03 GRPO steps and $16$04 candidates per step. RL stability depends on reward normalization and on a heuristic identification of $16$05 with negative flow matching loss. The adapted prefixes are per-speaker rather than globally reusable, and severe distribution shifts beyond the backbone’s linguistic support remain an open problem. The paper therefore presents VoiceTTA less as a replacement for strong zero-shot backbones than as a plug-in personalization layer for uncommon speaking styles (Xie et al., 25 Jun 2026).