Flamed-TTS: Zero-Shot Text-to-Speech

• Flamed-TTS is a zero-shot text-to-speech framework that integrates a discrete-code prior with continuous flow matching to generate speech that preserves speaker identity, prosody, and style.
• It employs an attention-free ConvNeXt denoiser and probabilistic duration and silence generators to reduce token repetition, lower latency, and enhance temporal diversity.
• The model achieves low WER and high intelligibility on LibriTTS and LibriSpeech by combining non-autoregressive code prediction with iterative acoustic refinement.

Flamed-TTS is a zero-shot text-to-speech framework introduced in "Flamed-TTS: Flow Matching Attention-Free Models for Efficient Generating and Dynamic Pacing Zero-shot Text-to-Speech" (Huynh-Nguyen et al., 3 Oct 2025). It is designed for the setting in which a system receives target text together with a short reference speech prompt from an unseen speaker and must synthesize speech that preserves speaker identity, prosody, and acoustic style without speaker-specific fine-tuning. The model is defined by four coupled design choices: a non-autoregressive discrete code prior, a continuous latent generator trained with conditional flow matching, an attention-free ConvNeXt-based denoiser, and probabilistic duration and silence generators for dynamic pacing. Within the paper’s experimental scope, this combination is presented as a way to reduce token-repetition failures and unexpected content transfer, lower inference cost and latency, and improve temporal diversity while maintaining high intelligibility and speech fidelity (Huynh-Nguyen et al., 3 Oct 2025).

1. Problem setting and conceptual position

Flamed-TTS addresses a specific tension in recent zero-shot TTS. Discrete-valued codec-token systems such as VALL-E and related work model tokenized speech sequences, often autoregressively, but are described as vulnerable to sampling errors such as token repetition. Continuous-valued systems based on diffusion or flow matching operate in mel or latent acoustic spaces and can use prompt speech for in-context conditioning, but the paper argues that they often remain computationally heavy and slow at inference. A further target is temporal diversity: the authors state that non-autoregressive systems usually rely on deterministic duration prediction and therefore under-model variability in phoneme durations and pauses (Huynh-Nguyen et al., 3 Oct 2025).

The proposed response is not to remove iterative generation altogether. Instead, Flamed-TTS reformulates flow matching so that denoising starts from a semantically enriched prior derived from predicted codec codes rather than from pure Gaussian noise. This shifts the denoiser’s function from global semantic construction toward local acoustic refinement. The title’s phrase “attention-free” is therefore specific: the paper removes multi-head self-attention from the denoiser and replaces it with a lightweight ConvNeXt-style module, rather than claiming that every part of the end-to-end system is attention-free (Huynh-Nguyen et al., 3 Oct 2025).

The paper presents three headline contributions. First, it introduces attention-free flow matching with a semantic prior. Second, it adds probabilistic duration and silence modeling for dynamic pacing. Third, it reports strong zero-shot performance under a compact, low-latency architecture, including a reported best WER of 4% relative to the listed baselines (Huynh-Nguyen et al., 3 Oct 2025).

2. Hybrid discrete-to-continuous architecture

The full synthesis pipeline contains a Code Generator, a Denoiser, and a frozen FACodec stack (Huynh-Nguyen et al., 3 Oct 2025). FACodec provides two representations of the reference speech. One is a continuous latent representation used as the target acoustic space. The other is a set of six discrete code sequences, specifically one prosody code stream, two content code streams, and three acoustic-detail code streams. These six streams function as disentangled speech attributes and form the basis of the model’s semantically enriched prior.

Target text is converted to phonemes and encoded by a Phoneme Encoder, yielding an encoded phoneme sequence

$\mathcal{P} \in \mathbb{R}^{L \times D}.$

This sequence is used by the duration and silence modules and by the code generation path. After temporal expansion, the Code Decoder predicts six target codec streams in a hierarchical manner. The factorization reported in the paper is

$$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$

Here, $\mathbf{q}_j$ is the predicted code sequence at hierarchy level $j$, $\mathbf{p}_j$ is the prompt-code sequence at the matching level, and $\mathcal{F}_{\psi}^j$ denotes the corresponding FFT decoder layers (Huynh-Nguyen et al., 3 Oct 2025).

Once the six code streams are predicted, Flamed-TTS applies the code encoding and folding mechanism from OZSpeech. The code tensor is reshaped from $B \times 6 \times L \times D$ to $B \times L \times 6D$ and then compressed by CNN layers into $B \times L \times D'$. This compressed representation is the semantic prior $\mathbf{x}_{pr}$ supplied to the continuous generator (Huynh-Nguyen et al., 3 Oct 2025).

This hybrid design is central to the paper’s identity. Discrete codes are used for alignment, attribute disentanglement, and prior formation, whereas the continuous FACodec latent remains the final target space for high-fidelity acoustic realization. A plausible implication is that Flamed-TTS is best understood neither as a pure codec LLM nor as a pure mel-latent flow model, but as a discrete-to-continuous zero-shot TTS architecture in which the discrete stage narrows the denoiser’s search space before continuous refinement (Huynh-Nguyen et al., 3 Oct 2025).

3. Reformulated flow matching and the attention-free denoiser

The paper first recaps standard optimal-transport conditional flow matching. With target sample $$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$0 and initial sample $$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$1, the interpolation is

$$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$2

and the target velocity is

$$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$3

The vector field estimator is trained with mean squared error against this velocity (Huynh-Nguyen et al., 3 Oct 2025).

Flamed-TTS modifies the initial point. Instead of pure noise, it uses

$$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$4

where $$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$5 is the semantically enriched prior, $$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$6, and $$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$7 is a noise scaling factor. The main flow-matching objective becomes

$$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$8

The model also uses an anchor loss,

$$p(\mathbf{q}_{1:6} \mid \mathcal{P}; \mathbf{p}; \psi) = p(\mathbf{q}_{1} \mid \mathcal{P}; \mathbf{p}_{1}; \mathcal{F}_{\psi}^1) \times \prod_{j=2}^{6} p(\mathbf{q}_{j} \mid \mathbf{q}_{j-1}; \mathbf{p}_{j}; \mathcal{F}_{\psi}^j).$$9

and the total objective is

$\mathbf{q}_j$0

These terms jointly optimize discrete-code prediction, duration generation, silence generation, and continuous latent refinement (Huynh-Nguyen et al., 3 Oct 2025).

Architecturally, the denoiser is a modified DiT in which multi-head self-attention is replaced by a ConvNeXt module. The stated computational motivation is the complexity change from

$\mathbf{q}_j$1

for self-attention to

$\mathbf{q}_j$2

for the ConvNeXt-based replacement, where $\mathbf{q}_j$3 is sequence length, $\mathbf{q}_j$4 is kernel size, and $\mathbf{q}_j$5 is hidden dimension (Huynh-Nguyen et al., 3 Oct 2025). The authors’ argument is that once semantic structure has already been injected through $\mathbf{q}_j$6, the denoiser no longer needs the same degree of global attention-based reasoning.

The paper also reports an ablation over the noise scale $\mathbf{q}_j$7. Best performance in the reported setting occurs at $\mathbf{q}_j$8 with a 5-second prompt and NFE 64, giving UTMOS 3.87, WER 0.04, SIM-O 0.51, and SIM-R 0.59. Larger $\mathbf{q}_j$9 degrades all metrics, which the paper interprets as evidence that excessive noise destroys the benefit of the learned prior (Huynh-Nguyen et al., 3 Oct 2025).

4. Dynamic pacing through probabilistic duration and silence generation

A second major contribution is explicit stochastic timing. Flamed-TTS does not rely on a deterministic duration predictor followed by a conventional length regulator. Instead, it introduces a Probabilistic Duration Generator and a Silence Generator, both trained with flow matching in the log-duration domain (Huynh-Nguyen et al., 3 Oct 2025).

The duration objective is

$j$0

with

$j$1

The silence objective is

$j$2

with

$j$3

At inference, both modules are sampled by ODE integration from Gaussian initial states (Huynh-Nguyen et al., 3 Oct 2025).

The supplementary algorithm specifies Euler updates,

$j$4

$j$5

with $j$6. Final phoneme durations are recovered as

$j$7

for $j$8, while silence durations are

$j$9

The paper prepends a special $\mathbf{p}_j$0 token and inserts silence after each phoneme; silence duration may be zero, while minimum phoneme duration is constrained to one frame (Huynh-Nguyen et al., 3 Oct 2025).

Empirically, the temporal-diversity analysis is one of the paper’s clearest differentiators. For 5-second prompts, Flamed-TTS reports Speech Rate $\mathbf{p}_j$1, MPhD $\mathbf{p}_j$2, $\mathbf{p}_j$3, and MPaD $\mathbf{p}_j$4. By contrast, the deterministic non-autoregressive baselines NaturalSpeech 2 and OZSpeech report about 1.20 and 1.18 pauses, respectively, with mean pause durations around 0.03 (Huynh-Nguyen et al., 3 Oct 2025). The paper explicitly interprets this as approximately 4× more pauses and 5× longer average pause duration than those baselines. This suggests that dynamic pacing is not treated as a cosmetic post-processing step but as a learned stochastic subproblem directly tied to the model’s naturalness claim.

5. Training setup, inference procedure, and reported performance

Within the reported experimental setup, Flamed-TTS is trained on LibriTTS, using 500 hours of speech, and evaluated on LibriSpeech test-clean with prompt lengths of 1 second, 3 seconds, and 5 seconds (Huynh-Nguyen et al., 3 Oct 2025). Baselines include Spark-TTS, VoiceCraft, NaturalSpeech 2, VALL-E, F5-TTS, and OZSpeech. The paper evaluates UTMOS, WER, SIM-O, SIM-R, pitch and energy preservation metrics, NFE, RTF, and temporal-diversity measures.

Inference proceeds in stages. FACodec encodes the prompt speech. The target text is phonemized and passed through the Phoneme Encoder. Duration and silence trajectories are sampled, producing an expanded phoneme sequence. The Code Decoder predicts the six discrete codec streams in one forward pass. These streams are folded and compressed into $\mathbf{p}_j$5. Noise is added to form $\mathbf{p}_j$6, and the attention-free flow-matching denoiser iteratively refines the latent representation. The final latent is then decoded by the FACodec decoder into waveform (Huynh-Nguyen et al., 3 Oct 2025). The paper does not specify the exact ODE solver for the main latent denoiser beyond the reported number of function evaluations.

The principal intelligibility result is the reported WER of 0.04. For 3-second prompts, Flamed-TTS reports WER 0.04, compared with 0.05 for OZSpeech, 0.09 for NaturalSpeech 2, 0.10 for Spark-TTS, 0.18 for VoiceCraft, 0.19 for VALL-E, and 0.24 for F5-TTS. For 5-second prompts, the reported WER remains 0.04, while OZSpeech is 0.05 and F5-TTS rises to 0.32 (Huynh-Nguyen et al., 3 Oct 2025). The abstract summarizes this as a best WER of 4% relative to the listed baselines.

Naturalness improves with additional denoising steps. For 5-second prompts, the NFE ablation reports UTMOS 3.13 at NFE 2, 3.49 at NFE 4, 3.68 at NFE 8, 3.79 at NFE 16, 3.84 at NFE 32, 3.87 at NFE 64, 3.88 at NFE 128, and 3.90 at NFE 256, while WER remains 0.04 from NFE 4 onward (Huynh-Nguyen et al., 3 Oct 2025). This is presented as evidence that intelligibility saturates early and naturalness continues to benefit from further refinement.

Latency and runtime are reported via RTF. For 3-second prompts, Flamed-TTS reports RTF 0.016 at 16 NFE, 0.028 at 32 NFE, 0.040 at 64 NFE, and 0.073 at 128 NFE. The paper states that Flamed-TTS at 32 steps is nearly 10× smaller in RTF than F5-TTS at the same step count, and the abstract claims up to 106× faster inference than competing baselines (Huynh-Nguyen et al., 3 Oct 2025). Model size is also part of the efficiency claim: Flamed-TTS uses 143M trainable parameters plus 102M frozen FACodec parameters, and a 76M-parameter variant reportedly preserves WER and speaker similarity while reducing UTMOS by about 4.5–6% (Huynh-Nguyen et al., 3 Oct 2025).

Speaker similarity is more mixed than intelligibility. For 5-second prompts, the reported SIM-O is 0.51, compared with 0.61 for Spark-TTS and 0.57 for F5-TTS, while SIM-R is 0.59, compared with 0.74 for Spark-TTS and 0.48 for OZSpeech (Huynh-Nguyen et al., 3 Oct 2025). Acoustic-characteristic preservation is stronger: for 5-second prompts, Flamed-TTS reports $\mathbf{p}_j$7, $\mathbf{p}_j$8, $\mathbf{p}_j$9, and $\mathcal{F}_{\psi}^j$0, which the paper identifies as the best values in the comparison (Huynh-Nguyen et al., 3 Oct 2025).

6. Interpretation, related systems, and limitations

Flamed-TTS occupies a distinct position among recent TTS systems. Relative to FireRedTTS-1S, which organizes streaming synthesis into text-to-semantic decoding followed by semantic-to-acoustic decoding with a multi-stream acoustic LLM and causal codec (Guo et al., 26 Mar 2025), Flamed-TTS replaces streaming autoregressive token generation with a hybrid non-autoregressive code prior and iterative continuous refinement (Huynh-Nguyen et al., 3 Oct 2025). Relative to FireRedTTS-2, which uses a 12.5 Hz streaming tokenizer and a dual-transformer architecture over text-speech interleaved dialogue context (Xie et al., 2 Sep 2025), Flamed-TTS is aimed at zero-shot monologic voice cloning rather than long conversational turn-by-turn speech generation. Relative to PFluxTTS, which fuses duration-guided and alignment-free vector fields at inference time and emphasizes cross-lingual cloning with prompt-token conditioning (Pankov et al., 4 Feb 2026), Flamed-TTS relies on a single attention-free denoiser with a semantically enriched prior and does not study cross-lingual or in-the-wild multilingual prompting in the reported experiments.

Several misconceptions are therefore worth excluding. Flamed-TTS is not a purely autoregressive codec LLM, because its code generation is non-autoregressive and its acoustic realization is flow-based. It is not a pure noise-to-speech flow-matching model either, because the initial state is learned from predicted FACodec codes rather than sampled only from $\mathcal{F}_{\psi}^j$1. It is also not a one-step generator in the general case: the paper explicitly studies multiple NFEs and shows that naturalness improves as more denoising evaluations are used (Huynh-Nguyen et al., 3 Oct 2025).

The paper is comparatively restrained about failure analysis, but several limitations are explicit or directly implied. Spark-TTS remains stronger in some settings for UTMOS and speaker similarity. Flamed-TTS remains iterative, so its best naturalness still depends on multiple denoising evaluations. The system depends on FACodec, so any limitations of that codec stack propagate into both the prior and the waveform decoder. The reported experiments are English-only, based on LibriTTS and LibriSpeech. The paper does not provide full training hyperparameters, does not specify the exact main ODE solver beyond NFE counts, and does not report human MOS listening tests in the provided content, relying instead on automatic metrics such as UTMOS, WER, speaker-similarity scores, and prosodic preservation measures (Huynh-Nguyen et al., 3 Oct 2025).

In that sense, Flamed-TTS is best characterized as a technically specific answer to a narrow but important design question in zero-shot TTS: whether a semantically structured discrete prior can make continuous flow matching cheap enough to dispense with attention in the denoiser while also recovering more human-like timing through probabilistic pacing. The reported results support that claim most strongly in intelligibility, efficiency, and temporal diversity, while leaving cross-lingual robustness, broader multilinguality, and human-subjective validation as open issues relative to adjacent systems (Huynh-Nguyen et al., 3 Oct 2025).