MagpieTTS-LF: Inference Long-Form Speech Synthesis

• MagpieTTS-LF is an inference-based method that retrofits encoder-decoder TTS systems to generate coherent, long-form speech by maintaining state across sentence chunks.
• It employs soft attention priors, stateful cross-chunk inference, and history-aware text encoding to preserve monotonic alignment and prosodic planning.
• Experimental evaluations show significant gains in intelligibility, speaker consistency, and prosodic continuity over naive chunk-and-concatenation approaches without requiring retraining.

Searching arXiv for the specified paper to ground the article in the current record. MagpieTTS-LF is an inference-time long-form speech generation method proposed to enable MagpieTTS, a standard short-utterance neural text-to-speech system, to synthesize coherent paragraph- or multi-minute-length speech without model retraining and without requiring long-form training data. It is presented as a retrofit for an already-trained encoder-decoder TTS model and is motivated by the claim that long-form failure in modern neural TTS is not solely a modeling-capacity problem but also an inference problem: if monotonic alignment is preserved, the relevant state is carried across sentence chunks, and limited text history is exposed for prosodic planning, a model trained only on normal utterance-length data can generate substantially more stable long-form speech (Ghosh et al., 16 Jun 2026).

1. Problem formulation and historical context

The paper situates MagpieTTS-LF against a practical limitation of contemporary neural TTS. Systems such as Tortoise, VALL-E 2, VALL-E R, NaturalSpeech, XTTS, Qwen3-TTS, MagpieTTS, and CosyVoice can produce very natural short utterances, roughly in the 2–20 second range, but they do not provide a native mechanism for paragraph-level synthesis (Ghosh et al., 16 Jun 2026). When longer passages are attempted, generation commonly falls back to sentence-level chunking plus concatenation, which produces several characteristic failure modes: prosodic drift over time, speaker inconsistency across segments, sentence boundary artifacts, hallucinations, loss of cross-sentence intonation patterns, energy discontinuities and warbles at boundaries, inconsistent speaking rate across chunks, and eventual intelligibility degradation as sequence length grows (Ghosh et al., 16 Jun 2026).

The paper identifies three broad prior paradigms for long-form TTS. Sequence compression methods increase effective context length by reducing token rate; the cited example is VibeVoice at 7.5 Hz, where one token covers about 133 ms of audio, allowing very long generation in one pass but, according to the authors, at the cost of temporal resolution that appears to hurt intelligibility and speaker preservation (Ghosh et al., 16 Jun 2026). Streaming or block-wise approaches such as CosyVoice 2 support chunk-wise generation with bounded memory through architectural changes and training-time masking schemes, but they require retraining and impose hard block boundaries via binary masks (Ghosh et al., 16 Jun 2026). Explicit cross-sentence prosody models such as HiGNN-TTS and context-aware memory systems improve discourse-level prosody through specialized graph or memory components trained jointly with the TTS model (Ghosh et al., 16 Jun 2026). MagpieTTS-LF is introduced as an alternative to all three patterns: an inference-only strategy for long-form synthesis that changes neither the trained architecture nor the training data regime (Ghosh et al., 16 Jun 2026).

This positioning is important because the method’s central thesis is not that chunking can be eliminated, but that chunking need not imply isolation. The paper argues that, with appropriate inference-time continuity mechanisms, chunk-wise generation can behave more like continuation of an ongoing utterance than repeated cold starts (Ghosh et al., 16 Jun 2026).

2. Base MagpieTTS architecture and the retrofit premise

MagpieTTS-LF is built on the base MagpieTTS architecture, which follows Koel-TTS and is described as an encoder-decoder Transformer operating on discrete audio tokens produced by a neural audio codec (Ghosh et al., 16 Jun 2026). The text encoder converts text into contextual hidden representations, and the decoder auto-regressively generates codec tokens while cross-attending to encoded text and, when used, audio context for voice cloning (Ghosh et al., 16 Jun 2026). During training, MagpieTTS already uses CTC loss and learned attention priors to encourage monotonic text-audio alignment and to reduce hallucinations such as skipping, repetition, and misalignment (Ghosh et al., 16 Jun 2026).

MagpieTTS-LF does not alter this trained architecture. Its interventions are strictly at inference time: how attention is biased, how long text is chunked and processed, and what state is carried from one chunk to the next (Ghosh et al., 16 Jun 2026). This distinction is fundamental. The method does not rely on long-form adaptation, long-form finetuning, or long-form data. It is therefore presented as attractive in deployment settings where a strong short-form encoder-decoder TTS system already exists and retraining is either impractical or undesirable (Ghosh et al., 16 Jun 2026).

The authors further state that the approach can extend to “any chunk-based encoder-decoder TTS system for long-form synthesis” (Ghosh et al., 16 Jun 2026). This suggests conceptual generality, although the same paper also makes clear that the method is most naturally applicable to models with decoder cross-attention over encoded text and a reasonably monotonic text-audio alignment structure. A plausible implication is that MagpieTTS-LF is best viewed not as a universally architecture-agnostic recipe, but as a class of inference strategies tailored to encoder-decoder TTS systems with monotonic cross-attention tendencies.

3. Core inference-time mechanisms

MagpieTTS-LF consists of three coupled mechanisms: soft attention priors, stateful cross-chunk inference, and history-aware text encoding (Ghosh et al., 16 Jun 2026). Each addresses a distinct failure mode of naive sentence-wise synthesis.

The first component is an inference-time soft attention prior. At each decoding step $t$, the method identifies a text position $T_t$, defined as the encoder position with the highest cross-attention score at decoding step $t-1$ (Ghosh et al., 16 Jun 2026). Around this position it constructs a local prior window while assigning all remaining positions a small non-zero probability mass. The paper defines the prior distribution $P_t \in \mathbb{R}^N$ as

$$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$

$$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$

with

$w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$

This prior is strongest at the current attended token and nearby future positions, but it preserves non-zero access to the full encoder context (Ghosh et al., 16 Jun 2026). The modified attention is written in the paper as

$\tilde{A}_t = \mathrm{softmax}\left(\frac{Q_t K^\top}{\sqrt{d} + \lambda \log P_t\right)$

with reported experimental setting $\lambda = 1.0$ (Ghosh et al., 16 Jun 2026). The same paper notes that the notation appears to place the prior inside the denominator term, while the intended interpretation is that the prior contributes a log-bias to the attention logits, “gently guiding” learned attention toward local monotonic alignment rather than enforcing it with a binary mask (Ghosh et al., 16 Jun 2026). That distinction is central: unlike streaming masks or hard chunk masks, the prior does not zero out distant positions.

The second component is the stateful chunk generation algorithm. Long text is first split into punctuation-aware sentences,

$S = \{s_0, s_1, \ldots, s_M\}.$

Generation then proceeds sentence by sentence with persistent state. The state contains three elements. First, history text tokens $T_t$0, defined as the final $T_t$1 text tokens from the previous chunk, are prepended to the current sentence (Ghosh et al., 16 Jun 2026). Second, history encoder context $T_t$2, the encoder hidden states corresponding to those history tokens, is carried over and concatenated with the current chunk’s encoded representations (Ghosh et al., 16 Jun 2026). Third, attention tracking preserves the last attended text positions from the previous chunk as an initial soft-attention prior $T_t$3, so the next chunk begins aligned where the previous one ended (Ghosh et al., 16 Jun 2026).

The resulting encoder input and effective encoder context are written as

$T_t$4

and

$T_t$5

The paper notes a specific ordering detail: history text tokens are appended first, the newly computed encoder outputs at those corresponding positions are discarded, and the cached history encoder context is appended instead (Ghosh et al., 16 Jun 2026). This means the current chunk’s text stream includes prior words for contextualization, but the hidden states for those words are reused from the previous chunk’s original encoding, which is intended to maintain continuity.

The third component is history-aware text encoding. The paper argues that long-form prosody depends not only on decoder continuity but also on the text encoder seeing some recent textual context so it can infer discourse relations and shape sentence-initial prosody appropriately (Ghosh et al., 16 Jun 2026). Rather than introducing a new planning module, the method simply prepends the last $T_t$6 text tokens from the previous chunk before encoding the current sentence. This uses the model’s native text encoder for “prosodic planning” by exposing discourse context through self-attention and by preserving continuity in the encoder hidden-state stream (Ghosh et al., 16 Jun 2026).

4. End-to-end generation procedure

Operationally, MagpieTTS-LF synthesizes long text sentence by sentence, but not independently. A long text passage is first cleaned and normalized and then segmented using punctuation-aware sentence splitting (Ghosh et al., 16 Jun 2026). The first sentence is synthesized with empty history or an initial neutral state. For each later sentence, the system prepends a carryover text window from the previous sentence, reuses cached encoder states for that carryover window, and initializes decoder cross-attention using the previous sentence’s ending alignment position (Ghosh et al., 16 Jun 2026).

During auto-regressive decoding, the local soft attention prior is applied at every step and updated from the latest attended text position, maintaining monotonic progression through the combined history-plus-current text (Ghosh et al., 16 Jun 2026). When the sentence reaches an end-of-speech token, the generated codec tokens are stored, the last $T_t$7 text tokens and corresponding encoder states are cached, and the final attention position is saved for the next sentence (Ghosh et al., 16 Jun 2026). After all sentences are processed, all per-sentence codec sequences are concatenated into one long codec sequence and decoded to waveform by the codec/vocoder path (Ghosh et al., 16 Jun 2026).

The paper gives enough detail to reconstruct the effective algorithm, even though it does not provide line-by-line pseudocode. The following summary condenses the procedure described in prose (Ghosh et al., 16 Jun 2026):

Step	Operation	Persistent state
1	Split long text into punctuation-aware sentence chunks	None
2	Initialize empty history and initial prior state $T_t$8	$T_t$9
3	For each $t-1$0, form $t-1$1, encode, build $t-1$2, decode with soft prior, update prior each step, stop at EOS	Updated after sentence
4	Cache final $t-1$3 text tokens, encoder states, and attention-tracking state; append codec tokens	$t-1$4
5	Concatenate all codec chunks	Final code stream

A notable implementation point is that the paper does not describe overlap-add waveform stitching or explicit crossfading between audio chunks (Ghosh et al., 16 Jun 2026). Continuity is instead achieved upstream at the token-generation stage, so simple concatenation of generated codes is reported to suffice. Because the method remains chunk-wise, the paper frames it as avoiding the memory overhead of single-pass long-context generation, although it does not provide explicit latency or throughput tables (Ghosh et al., 16 Jun 2026).

5. Experimental evaluation and reported results

Evaluation is conducted on a curated English long-form dataset comprising 20 long texts and what the paper calls a Long-Form HifiTTS dataset/benchmark (Ghosh et al., 16 Jun 2026). The benchmark is described as 20 passages of approximately 3–4 minutes each, constructed by concatenating paragraphs from Multilingual LibriSpeech, with text normalized and cleaned and duration estimated assuming 135 words per minute (Ghosh et al., 16 Jun 2026). The paper also reports evaluation on a “long-form HiFiTTS 1-hour subset” in the intelligibility table caption, which it acknowledges only indirectly through inconsistent naming (Ghosh et al., 16 Jun 2026). Baselines are Qwen3-TTS, VibeVoice-TTS, and XTTS, with all inference run on a single NVIDIA A6000 GPU (Ghosh et al., 16 Jun 2026).

The evaluation is objective and proxy-subjective rather than MOS-based. Intelligibility and alignment robustness are measured with WER and CER from Whisper-Large transcriptions. Speaker similarity is measured as cosine similarity between reference-speaker and generated-speech embeddings using both TitaNet and WavLM. Boundary prosody is assessed with Prosodic Boundary Discontinuity metrics based on $t-1$5 ms windows around sentence boundaries, specifically $t-1$6F0 and $t-1$7Energy, aggregated per passage and min-max normalized into a composite score. Naturalness consistency over time is assessed with UTMOSv2 on non-overlapping 10-second windows, and speaker consistency over time is measured analogously on 10-second windows as a function of relative utterance position (Ghosh et al., 16 Jun 2026).

The main reported quantitative results are as follows (Ghosh et al., 16 Jun 2026):

System	Intelligibility / speaker similarity	Boundary prosody
MagpieTTS-LF	WER $t-1$8, CER $t-1$9, SSIM(TitaNet) $P_t \in \mathbb{R}^N$0, SSIM(WavLM) $P_t \in \mathbb{R}^N$1	$P_t \in \mathbb{R}^N$2F0 $P_t \in \mathbb{R}^N$3, $P_t \in \mathbb{R}^N$4Energy $P_t \in \mathbb{R}^N$5, Composite $P_t \in \mathbb{R}^N$6
XTTS	WER $P_t \in \mathbb{R}^N$7, CER $P_t \in \mathbb{R}^N$8, SSIM(TitaNet) $P_t \in \mathbb{R}^N$9, SSIM(WavLM) $$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$0	$$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$1F0 $$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$2, $$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$3Energy $$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$4, Composite $$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$5
Qwen3-TTS	WER $$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$6, CER $$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$7, SSIM(TitaNet) $$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$8, SSIM(WavLM) $$P_t[i] = w_j , \quad \text{if } i \in \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$9	$$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$0F0 $$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$1, $$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$2Energy $$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$3, Composite $$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$4
VibeVoice	WER $$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$5, CER $$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$6, SSIM(TitaNet) $$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$7, SSIM(WavLM) $$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$8	$$P_t[i] = eps, \quad \text{if } i \notin \{T_t-1, T_t, T_t+1, T_t+2, T_t+3 \}$$9F0 $w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$0, $w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$1Energy $w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$2, Composite $w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$3

These figures underpin the paper’s principal claim that MagpieTTS-LF improves long-range intelligibility, speaker consistency, and sentence-boundary naturalness without retraining (Ghosh et al., 16 Jun 2026). The paper emphasizes that $w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$4F0 is relatively similar across systems, in the 65–69 Hz range, whereas $w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$5Energy differentiates methods more strongly: MagpieTTS-LF reduces energy discontinuity to 14.04 dB, approximately half of XTTS and substantially lower than VibeVoice (Ghosh et al., 16 Jun 2026). The longitudinal analyses reported in the figures likewise show MagpieTTS-LF maintaining the highest and most stable speaker similarity and UTMOSv2 trajectories across utterance position, while competing systems exhibit higher variance or visible drift (Ghosh et al., 16 Jun 2026).

6. Interpretation, scope, and limitations

The paper attributes its gains component-wise. The soft attention prior is said to improve alignment robustness over long horizons by preventing the decoder from drifting too far from the monotonic text path while preserving access to long-range context (Ghosh et al., 16 Jun 2026). Stateful inference is said to reduce sentence-boundary resets because each new chunk begins from the previous chunk’s ending attention region rather than from a cold start (Ghosh et al., 16 Jun 2026). History-aware text encoding is said to improve discourse-level prosodic planning because the encoder sees recent text rather than only the current isolated sentence (Ghosh et al., 16 Jun 2026). Together, these mechanisms are presented as explaining the observed improvements in intelligibility, prosodic continuity, and speaker consistency (Ghosh et al., 16 Jun 2026).

At the same time, the paper does not provide a dedicated ablation table removing each of the three components individually (Ghosh et al., 16 Jun 2026). It repeatedly attributes specific behaviors to each component, and the naive chunking baselines provide indirect evidence, but controlled component-wise quantitative isolation is absent. This is a limitation of the experimental analysis rather than of the method description itself.

Several additional limitations are explicitly acknowledged or visible in the presentation. There is no human listening study; evaluation relies on ASR-derived intelligibility, embedding-based speaker similarity, Prosodic Boundary Discontinuity metrics, and UTMOSv2 (Ghosh et al., 16 Jun 2026). The method assumes a sentence-segmentable text structure and a chunk-based encoder-decoder TTS model with monotonic cross-attention, so applicability to radically different architectures is not demonstrated (Ghosh et al., 16 Jun 2026). The approach uses several inference hyperparameters—$w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$6, $w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$7, $w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$8, temperature $w = (0.2, 0.8, 1.0, 0.8, 0.2), \qquad eps = 0.1.$9, CFG scale $\tilde{A}_t = \mathrm{softmax}\left(\frac{Q_t K^\top}{\sqrt{d} + \lambda \log P_t\right)$0, and history length $\tilde{A}_t = \mathrm{softmax}\left(\frac{Q_t K^\top}{\sqrt{d} + \lambda \log P_t\right)$1—but their sensitivity is not studied in detail (Ghosh et al., 16 Jun 2026). The attention-bias equation is also described as notationally awkward, and some dataset naming is inconsistent, which means implementers may need to infer intended details rather than directly copying every expression (Ghosh et al., 16 Jun 2026).

In deployment terms, the reported advantages are deployment simplicity, preservation of the original model, no need for architecture changes, no need for special long-form corpora, bounded memory due to chunk-wise generation, and substantially better long-range coherence than naive sentence concatenation (Ghosh et al., 16 Jun 2026). The exact reported inference hyperparameters are

$\tilde{A}_t = \mathrm{softmax}\left(\frac{Q_t K^\top}{\sqrt{d} + \lambda \log P_t\right)$2

with sampling temperature $\tilde{A}_t = \mathrm{softmax}\left(\frac{Q_t K^\top}{\sqrt{d} + \lambda \log P_t\right)$3 and classifier-free guidance scale $\tilde{A}_t = \mathrm{softmax}\left(\frac{Q_t K^\top}{\sqrt{d} + \lambda \log P_t\right)$4 (Ghosh et al., 16 Jun 2026). In that sense, MagpieTTS-LF defines a specific inference regime for extending a short-form encoder-decoder TTS model into long-form generation, rather than a new end-to-end training paradigm.

7. Significance within long-form neural TTS

MagpieTTS-LF’s significance lies in its reformulation of long-form TTS as an inference problem that can be partially solved without retraining (Ghosh et al., 16 Jun 2026). The method does not claim to remove all long-form challenges, nor does it replace architectural approaches that expand context length or introduce explicit discourse modeling. Instead, it demonstrates that preserving alignment state, encoder continuity, and limited textual history can materially improve long-form synthesis even when the base model was trained only on ordinary utterance-length data (Ghosh et al., 16 Jun 2026).

Within the broader development of neural TTS, this places MagpieTTS-LF between naive chunk-and-concatenate systems and retraining-heavy long-context or streaming systems. It retains chunk-wise generation for memory efficiency, but attempts to eliminate the principal liabilities of chunk independence. The paper’s results suggest that, for encoder-decoder TTS systems with monotonic alignment tendencies, long-form quality depends not only on extending context windows or compressing token streams, but also on how inference handles continuity across sentence boundaries (Ghosh et al., 16 Jun 2026). A plausible implication is that future long-form TTS systems may increasingly combine architecture-level scaling with stateful, alignment-aware inference strategies of the kind MagpieTTS-LF formalizes.