VoiceFlow: Rectified Flow Matching in TTS

• The paper introduces a novel TTS approach that straightens the synthesis trajectory using rectified flow matching to drastically reduce sampling steps while ensuring quality.
• It leverages a conditional ODE formulation to transform Gaussian noise into mel-spectrogram features, enabling rapid and efficient acoustic modeling compared to traditional diffusion methods.
• Empirical benchmarks indicate that VoiceFlow outperforms conventional methods on datasets like LJSpeech by maintaining high MOS and low MCD with as few as 2-10 integration steps.

VoiceFlow is a text-to-speech (TTS) acoustic modeling approach based on rectified flow matching, which reformulates mel-spectrogram generation as integration of a conditional ordinary differential equation (ODE) between Gaussian noise and speech features. Unlike conventional diffusion models that rely on score estimation and numerous sampling iterations, VoiceFlow leverages an ODE-based trajectory, applying a rectified flow technique that straightens the synthesis path to yield high fidelity speech with minimal sampling steps. The framework demonstrates effective acceleration in TTS synthesis, maintaining or surpassing diffusion model performance under strict efficiency constraints.

1. Conditional ODE Formulation for TTS

VoiceFlow specifies mel-spectrogram generation as transport between distributions via a conditional ODE. Given $x_0\sim\mathcal{N}(0,I)$ (initial noise) and $x_1\sim\mathcal{P}_{\text{mel}}(\cdot|y)$ (ground-truth mel features conditioned on text $y$), the system models a time-indexed signal $x_t\in\mathbb{R}^d$ progressing from $t=0$ to $t=1$ as

$\frac{dx_t}{dt} = f_\theta(x_t, t \mid y)$

where $y$ represents frame-level phone embeddings, obtained via duration expansion of text. This formulation contrasts with stochastic differential equations (SDEs) as used in score-based diffusion models, resulting in deterministic integration paths.

To match the conditional flow, VoiceFlow trains $f_\theta$ such that, for a sampled $x_0$ and $x_1$, and an intermediate $t \in [0, 1]$, the point $x_t\sim\mathcal{N}(t x_1 + (1-t) x_0, \sigma^2 I)$ should be flowed in the direction $x_1-x_0$. The core flow-matching loss is

$$L_{FM} = \mathbb{E}_{t, x_0, x_1, x_t} \left\| f_\theta(x_t, t \mid y) - (x_1-x_0) \right\|^2$$

ensuring alignment with the vector field of the analytic linear Gaussian transport.

2. Rectified Flow Matching: Theory and Practice

Efficient inference in ODE-based TTS requires that the learned vector field yields nearly straight sampling trajectories. Naively trained models often produce curved flows, demanding a large number of solver steps to achieve fidelity. VoiceFlow introduces the rectified flow matching extension to address this.

The rectification procedure consists of:

1. Sampling a new noise $x_0'$ and generating $\hat{x}_1$ via ODE integration from $x_0'$, using the current $f_\theta$.
2. Treating $(x_0', \hat{x}_1)$ as initial and terminal endpoints, and training $f_\theta$ anew to map the straight vector $\hat{x}_1-x_0'$ over samples from the interpolated path $$x_t\sim\mathcal{N}(t\hat{x}_1+(1-t)x_0', \sigma^2 I)$$.

The corresponding rectified-flow loss is

$$L_{\text{ReFlow}} = \mathbb{E}_{t, x_0', \hat{x}_1, x_t} \left\| f_\theta(x_t, t \mid y) - (\hat{x}_1-x_0')\right\|^2$$

This post-training pass explicitly corrects the network's own endpoints, reducing trajectory curvature and enabling efficient Euler-type sampling with very few steps.

3. Sampling Algorithms and Computational Efficiency

VoiceFlow’s synthesis requires only numerical integration of the learned field $f_\theta$. The simplest method is Euler integration:

initialize x ← Normal(0,I) for k in 0 … N−1: t ← k / N x ← x + (1/N) * f_θ(x, t | y) return x

Here, $N$ is the target number of steps, $y$ is the text condition, and $f_\theta$ is the trained vector field. The linearization induced by rectified flow allows the process to operate with $N=2-10$ steps. This sharp reduction in required function evaluations directly improves runtime and resource consumption, especially when compared to diffusion models (e.g., GradTTS) that require $50-100$ stochastic passes for comparable results.

A plausible implication is that such efficient ODE integration enables deployment in latency-sensitive or resource-constrained environments without high-quality degradation.

4. Empirical Benchmarks and Comparative Analysis

Empirical results on single- and multi-speaker datasets (LJSpeech, LibriTTS) demonstrate VoiceFlow’s effectiveness:

Steps	MOS (GradTTS)	MOS (VoiceFlow)	MOSNet (trend)	MCD (VF<GT)
2	2.98/2.52	3.92/3.81	VF ≈ GT, GT > DT	VF best
10	3.97/3.43	4.10/3.84	VF ≈ GT, GT > DT	VF best
100	4.03/3.45	4.17/3.85	VF ≈ GT, GT > DT	VF best
Vocoded GT	4.52/4.42	—	—	—

With ground-truth durations and identical U-Net + HiFi-GAN backbone architectures, VoiceFlow consistently matches or surpasses GradTTS in both subjective mean opinion score (MOS) and objective metrics such as mel-cepstral distortion (MCD), particularly in the critically constrained $N\leq10$ regime. MOSNet scores confirm that VoiceFlow approaches ground-truth region even with minimal steps, unlike GradTTS, which degrades rapidly. F0 RMSE was not reported in the referenced work.

5. Ablation Studies and Trajectory Visualization

Ablation analyses isolate the impact of rectified flow. Omitting the rectification retraining (“–ReFlow”) causes significant degradation: with $N=2$ steps, CMOS drops $-0.78\pm0.13$ (LJSpeech) and $-1.21\pm0.19$ (LibriTTS) relative to the full model. Visualization of the ODE solution in 2-D projections shows that original flow-matching yields curved paths, while rectification produces nearly straight-line trajectories, confirming that synthesis efficiency is contingent on trajectory linearity.

This suggests that the primary bottleneck in ODE-based TTS models is not expressivity of the vector field but the alignment of the learned sampling path with the analytic transport.

6. Extensions, Practical Deployment, and Context

The rectified flow paradigm introduced in VoiceFlow has since informed further development in the field, including parameter-efficient versions such as SlimSpeech (Wang et al., 10 Apr 2025) that exploit similar techniques with aggressive model slimming and flow-distillation, achieving high-quality synthesis in a single Euler step. VoiceFlow’s straightforward solver, low evaluation count, and trajectory straightness make it suitable for edge device applications and real-time TTS.

Key considerations for practitioners include:

• Solver selection: Euler integration suffices when trajectories are nearly linear; more sophisticated solvers offer diminishing returns beyond $N\geq4$.
• Training overhead: Rectification is an offline retraining cost, not required at inference.
• Model architecture: U-Net backbone and HiFi-GAN vocoder are compatible, but trajectory straightness remains paramount under aggressive downscaling.

VoiceFlow’s ODE-based rectified flow matching framework constitutes a significant direction for efficient, high-fidelity neural speech synthesis. The methodology emphasizes the importance of synthesis trajectory regularization, resource minimization, and empirical validation against state-of-the-art baselines.