FlashTTS: Real-Time Streaming TTS

• FlashTTS is an LLM-based text-to-speech framework that achieves low latency and native streaming support in real-time dialogue systems.
• It employs a lagged multi-track input, parallel multi-token prediction (MTP), and an X-pred mean flow decoder to minimize buffering and accelerate acoustic decoding.
• Empirical results show FlashTTS attains a first-packet latency of 325 ms while enhancing throughput and maintaining zero-shot voice cloning and cross-lingual intelligibility.

FlashTTS is an LLM-based text-to-speech framework for real-time dialogue that targets two constraints that are often only partially addressed in recent TTS systems: low latency and native support for streaming inputs and outputs. Its defining design combines a lagged multi-track input representation, parallel Multi-Token Prediction (MTP), and an X-pred mean flow matching token-to-mel decoder, so that both input-side buffering and output-side acoustic decoding are reduced. In the reported operating point, FlashTTS achieves a First-Packet Latency of $325$ ms while preserving zero-shot voice cloning and cross-lingual intelligibility (Xie et al., 8 Jun 2026).

1. Problem formulation and motivation

FlashTTS is situated in the regime of speech dialogue systems in which text arrives incrementally and synthesized speech must begin as early as possible. The paper identifies two primary latency bottlenecks in prior LLM-based TTS systems: slow autoregressive acoustic token generation and slow multi-step flow matching in token-to-mel or token-to-wave stages. It also argues that many such systems are not natively streaming on the input side, because they still wait for a full sentence or a substantial text chunk before stable synthesis begins (Xie et al., 8 Jun 2026).

A central term in this context is sentence-level buffering: waiting until the whole sentence, or a large part of it, is available before acoustic modeling begins. FlashTTS avoids this through a stacked, lagged multi-track representation in which text, language conditioning, and speech generation are represented on parallel tracks. The paper explicitly contrasts this with a baseline behavior in which CosyVoice2 buffers 5 text tokens before synthesis, whereas FlashTTS can begin from a single token. This design is not merely a decoding optimization; it is an attempt to make the entire TTS stack behave as a native streaming component in a conversational pipeline (Xie et al., 8 Jun 2026).

This positioning distinguishes FlashTTS from systems whose main contribution is throughput on complete utterances. It is primarily a streaming-first system rather than a purely offline low-RTF system. A plausible implication is that its design priorities are better understood through first-packet and first-token latency than through end-to-end throughput alone.

2. Core architecture and signal flow

FlashTTS has a two-stage synthesis pipeline. A decoder-only transformer based on Qwen2.5-0.5B consumes streaming text, language information, and speaker conditioning, and produces a single stream of speech tokens. An X-pred MeanFlow decoder then maps these tokens to mel spectrograms, and a HiFi-GAN 24 kHz vocoder reconstructs the waveform (Xie et al., 8 Jun 2026).

Component	Configuration	Function
LLM backbone	Qwen2.5-0.5B; 24 layers; hidden size 896; 14 attention heads; feed-forward dimension 4864	Predicts speech tokens from lagged multi-track inputs
MTP modules	Linear projection + Qwen2.5-style decoder block	Predict several future speech tokens in parallel
Token-to-mel decoder	16-layer Diffusion Transformer; hidden size 768; about 159.25M parameters	Converts speech tokens to mel via X-pred MeanFlow
Vocoder	HiFi-GAN 24 kHz; about 50M parameters	Converts mel to waveform

The lagged multi-track architecture is the central systems-level idea. The speech track starts with a speaker embedding and then carries generated speech tokens. The text track receives streaming text tokens and uses padding after text ends to preserve alignment. The language track provides persistent language conditioning. The tracks are stacked in parallel and temporally lagged so that speech prediction can depend on earlier text context without requiring the full future sentence (Xie et al., 8 Jun 2026).

The speech token representation is provided by S3Tokenizer v2, which the paper describes as using ASR-supervised speech tokens that are highly stable and easier for the LLM to predict than purely acoustic codec tokens. This places FlashTTS in the single-stream semantic/acoustic token paradigm: the LLM predicts a stable speech-token sequence, while finer acoustic realization is deferred to the token-to-mel decoder (Xie et al., 8 Jun 2026).

3. Multi-Token Prediction and autoregressive acceleration

FlashTTS uses Multi-Token Prediction to reduce the sequential bottleneck of one-token-at-a-time autoregression. Given backbone hidden states $\mathbf{h}_{0:t}^{0}$, FlashTTS adds parallel MTP branches,

$$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$

where $k \in \{1,2,\dots,N-1\}$ indexes future-token branches. Each branch is trained to predict the $(k+1)$-th future token. The paper states that the MTP loss is cross-entropy over shifted targets and evaluates MTP-3 and MTP-5, with MTP-3 as the preferred operating point (Xie et al., 8 Jun 2026).

The MTP modules are deliberately lightweight: each contains one linear projection layer and one Qwen2.5-style decoder block. The backbone is frozen during Stage 2 MTP training, and the LM head is shared. This makes MTP a speculative-style acceleration layer rather than a replacement for the backbone’s main predictive distribution (Xie et al., 8 Jun 2026).

The paper also describes a verification operation inspired by Llasa+, in which MTP modules propose speculative future tokens and the frozen backbone’s stronger probability distributions are used to validate them. However, the exact speculative acceptance rule is not provided. This omission matters because it makes the precise decoding policy less reproducible than the model architecture itself (Xie et al., 8 Jun 2026).

The reported trade-off is explicit. MTP-3 improves latency substantially relative to Stage 1 alone, whereas MTP-5 adds little speed but harms quality. In the main latency table, MTP-3 with 2-NFE reaches TPS $73$, LLM latency $195$ ms, First-Packet Latency $325$ ms, and RTF $0.632$, whereas MTP-5 with 2-NFE reaches TPS $75$ and RTF $\mathbf{h}_{0:t}^{0}$0 but degrades to WER $\mathbf{h}_{0:t}^{0}$1, SIM $\mathbf{h}_{0:t}^{0}$2, and CMOS $\mathbf{h}_{0:t}^{0}$3 (Xie et al., 8 Jun 2026). One common misconception is therefore that more parallel speculative prediction is always better; the reported results show that aggressive MTP introduces a measurable quality penalty.

4. X-pred MeanFlow token-to-mel decoding

The second acceleration mechanism is the X-pred mean flow matching decoder. FlashTTS does not directly predict a velocity field in mel space. Instead, it predicts clean data $\mathbf{h}_{0:t}^{0}$4, i.e. the target mel spectrogram, and analytically derives the mean velocity estimate from that prediction. The average velocity over interval $\mathbf{h}_{0:t}^{0}$5 is defined as

$\mathbf{h}_{0:t}^{0}$6

with the mean-flow identity

$\mathbf{h}_{0:t}^{0}$7

and

$\mathbf{h}_{0:t}^{0}$8

Using the substitution

$\mathbf{h}_{0:t}^{0}$9

the paper writes the training target as

$$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$0

The X-pred parameterization is

$$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$1

from which the mean velocity estimate is derived as

$$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$2

The distillation loss is

$$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$3

At the boundary condition $$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$4, the paper states that this reduces to standard conditional flow matching loss (Xie et al., 8 Jun 2026).

The practical significance of this formulation is that FlashTTS reports high-fidelity token-to-mel generation in exactly 2 function evaluations. This is the key contrast with multi-step flow-matching baselines such as the 10-NFE CosyVoice2 comparator in the latency table. The paper also provides the 1-NFE reconstruction rule,

$$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$5

although the preferred operating points are 2-NFE and 3-NFE (Xie et al., 8 Jun 2026).

The decoder additionally uses block-level chunked attention, described as being inspired by block-wise streaming flow matching, so that token chunks can be transformed into mel chunks progressively. This suggests that the decoder is not merely low-step but also structured for streaming output rather than full-utterance batch decoding.

5. Training configuration and empirical performance

FlashTTS is trained on about 300,000 hours of open-source speech data: Emilia, Emilia-Yodas, LibriHeavy, and WenetSpeech4TTS. Stage 1 uses a dynamic frame-based batch size of 40,000 on 8 A100 GPUs with AdamW, peak learning rate $$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$6, 20k warmup steps, and 1M total steps. Stage 2 freezes the backbone and trains only the MTP modules on 4 A100 GPUs with peak learning rate $$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$7. X-pred MeanFlow distillation is performed on 8 RTX 4090 GPUs with batch size 2000, gradient accumulation 2, and peak learning rate $$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$8 (Xie et al., 8 Jun 2026).

The main latency comparison is against CosyVoice2 on a single NVIDIA RTX 4090, with the upstream text stream generated dynamically by Qwen2-7B to simulate realistic dialogue. The reported operating points are as follows (Xie et al., 8 Jun 2026).

Configuration	Latency/throughput	Quality
CosyVoice2 (10-NFE)	TPS 51; FTL 257 ms; FPL 843 ms; RTF 0.913	WER 26.2; SIM 0.721
Stage 1 (2-NFE)	TPS 50; FTL 60 ms; FPL 377 ms; RTF 0.793	WER 18.0; SIM 0.702
MTP-3 (2-NFE)	TPS 73; FTL 62 ms; FPL 325 ms; RTF 0.632	WER 18.8; SIM 0.695
MTP-3 (3-NFE)	TPS 72; FTL 62 ms; FPL 366 ms; RTF 0.702	WER 17.5; SIM 0.714

These results establish the paper’s main claim: input-side latency falls from $$\mathbf{h}_{0:t}^{k} = \text{MTP}_k(\mathbf{h}_{0:t}^{0}),$$9 ms to about $k \in \{1,2,\dots,N-1\}$0–$k \in \{1,2,\dots,N-1\}$1 ms, and first-packet latency falls from $k \in \{1,2,\dots,N-1\}$2 ms to $k \in \{1,2,\dots,N-1\}$3 ms at the preferred MTP-3, 2-NFE setting. The trade-off is that 3-NFE improves WER, SIM, and CMOS slightly, but at higher latency (Xie et al., 8 Jun 2026).

The broader multilingual evaluation uses the Seed-TTS evaluation set and the MiniMax multilingual test set. On MiniMax, FlashTTS reports zh WER $k \in \{1,2,\dots,N-1\}$4, SIM $k \in \{1,2,\dots,N-1\}$5; en WER $k \in \{1,2,\dots,N-1\}$6, SIM $k \in \{1,2,\dots,N-1\}$7; ja WER $k \in \{1,2,\dots,N-1\}$8, SIM $k \in \{1,2,\dots,N-1\}$9; ko WER $(k+1)$0, SIM $(k+1)$1; fr WER $(k+1)$2, SIM $(k+1)$3; and de WER $(k+1)$4, SIM $(k+1)$5. The paper interprets this as improved English and Korean intelligibility relative to CosyVoice2, together with French and German coverage where CosyVoice2 has no reported results (Xie et al., 8 Jun 2026).

The ablation study reinforces the architectural division of labor. Relative to CosyVoice2, the full FlashTTS model reports WER $(k+1)$6, SIM $(k+1)$7, and Speed-Up Ratio $(k+1)$8. Removing X-pred reduces the speed-up to $(k+1)$9; removing MTP reduces it to $73$0; removing Language ID preserves speed but degrades WER to $73$1 (Xie et al., 8 Jun 2026). This makes MTP and X-pred the two primary acceleration engines, while the language track is chiefly a robustness mechanism.

6. Position within fast and streaming TTS research

FlashTTS belongs to a recent cluster of systems that treat low latency as a first-class design objective, but it occupies a specific niche within that landscape. VoXtream and VoXtream2 are full-stream autoregressive systems that begin speaking from the first word and emphasize monotonic alignment and streaming codec generation; they report first-packet latencies of $73$2 ms on A100 with torch.compile and $73$3 ms on RTX3090, respectively, but use temporal/depth transformer codec-token factorization rather than an LLM plus low-NFE token-to-mel flow decoder (Torgashov et al., 19 Sep 2025, Torgashov et al., 13 Mar 2026). Chatterbox-Flash takes a different route by converting a pretrained autoregressive codec decoder into a block-diffusion decoder with prior-calibrated scoring and adaptive early decoding, reporting TTFP $73$4 ms and RTF $73$5 in its default streaming configuration (Seo et al., 29 May 2026).

Outside the strictly streaming-first setting, FlashTTS also differs from non-streaming efficient TTS systems that prioritize offline throughput or robustness. PFluxTTS uses hybrid inference-time vector-field fusion for robust cross-lingual voice cloning and reports RTF $73$6 on an NVIDIA A10 GPU, but it is not presented as a latency-optimization or streaming paper (Pankov et al., 4 Feb 2026). Flamed-TTS removes self-attention from the flow-matching denoiser, reports RTF $73$7 at 16 NFE, and emphasizes zero-shot quality and dynamic pacing rather than native streaming input/output (Huynh-Nguyen et al., 3 Oct 2025). This suggests that FlashTTS should be located within the streaming-first branch of efficient TTS research rather than treated as directly interchangeable with low-RTF offline systems.

Several limitations are explicit or visible in the reported results. First, aggressive acceleration degrades quality: MTP-5 is faster in TPS but negative in CMOS (Xie et al., 8 Jun 2026). Second, the exact speculative verification algorithm is omitted, and the lag schedule, chunk sizes, and masking implementation are not fully specified (Xie et al., 8 Jun 2026). Third, while FlashTTS is competitive in intelligibility, its similarity trails some larger or offline systems on parts of the evaluation suite (Xie et al., 8 Jun 2026). A common overstatement is therefore to treat FlashTTS as simply “the fastest TTS model”; the paper’s actual contribution is narrower and more technical. It is a joint redesign of input representation, autoregressive token generation, and low-step acoustic decoding for real-time streaming dialogue.

In that sense, FlashTTS marks a transition from streaming-adapted offline synthesis toward a genuinely native-streaming LLM-based TTS stack. Its most characteristic ideas are the lagged multi-track input representation, speculative multi-token speech-token generation, and X-pred mean flow distillation for 2-NFE token-to-mel decoding (Xie et al., 8 Jun 2026).