FlowSE: Flow Matching Speech Enhancement

• FlowSE is a family of flow-based speech enhancement methods that recast denoising as continuous transport between probability distributions.
• The approach leverages both conditional flow matching on STFT features and latent mel-spectrogram frameworks to achieve high quality with low inference steps.
• Empirical results show that FlowSE can match or exceed conventional methods while reducing computational latency by more than 10×.

Searching arXiv for FlowSE-related papers to ground the article in current research. Querying the arXiv API for papers matching “FlowSE” and closely related speech-enhancement work. $s$33

Running the arXiv API query now. $s$34

FlowSE is the name used for a set of speech enhancement methods that formulate denoising as continuous transport between probability distributions, learned through flow matching rather than iterative reverse diffusion. In these systems, a neural network estimates a velocity field on a continuous path connecting a simple source distribution to a clean-speech target, and enhancement is obtained by numerically integrating an ordinary differential equation with relatively few function evaluations. The designation covers at least two distinct 2025 formulations—one framed as conditional flow matching for speech enhancement, and another framed as an efficient mel-spectrogram generator with optional transcript conditioning—and it is subsequently extended by online reinforcement-learning post-training and contrasted with a time-unconditional autonomous rectified-flow alternative (Lee et al., 9 Aug 2025, Wang et al., 26 May 2025, Wang et al., 23 Jan 2026, Zhang et al., 18 Jun 2026).

1. Scope and nomenclature

The 2025 paper "FlowSE: Flow Matching-based Speech Enhancement" defines FlowSE as a conditional continuous normalizing flow for recovering clean speech from noisy observations, with emphasis on low-NFE inference without any additional fine-tuning procedure (Lee et al., 9 Aug 2025). The later 2025 paper "FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matching" uses the same name for a mel-spectrogram-based speech enhancement framework composed of a mel encoder, a latent Diffusion Transformer, and a neural vocoder, optionally conditioned on character transcripts (Wang et al., 26 May 2025).

A concise summary of the main variants is useful because the shared name masks substantial methodological differences.

Work	Core formulation	Reported emphasis
FlowSE (Lee et al., 9 Aug 2025)	Conditional flow matching / conditional CNF	Comparable quality at NFE $=5$ without extra fine-tuning
FlowSE (Wang et al., 26 May 2025)	Flow matching on mel spectrograms with optional text	High quality, low latency, real-time factor reporting
FlowSE-GRPO (Wang et al., 23 Jan 2026)	Online RL post-training of FlowSE	Metric alignment and reward-design analysis
Autonomous Rectified Flow (Zhang et al., 18 Jun 2026)	Time-unconditional rectified flow	Challenges explicit time conditioning

The shared conceptual core is a deterministic transport view of speech enhancement: rather than simulating a long stochastic reverse process, the model learns a vector field that maps noisy or latent initial states toward clean speech. A plausible implication is that "FlowSE" is best understood not as a single architecture, but as a family of flow-based SE formulations occupying a common design space.

2. Mathematical formulations

In the conditional-CNF formulation, the task is posed as recovering clean speech $s$ from a noisy observation $y=s+n$ by learning a continuous-time flow

$$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$

with $x_1 \approx q(s\mid y)$. Training uses the Conditional Flow Matching loss

$$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$

For the Gaussian conditional path

$$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$

the target field is given in closed form by

$$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$

The same work states that the corresponding diffusion model can be reinterpreted with drift $f(x,y,t)=(y-x)/(1-t)$ and diffusion $g(t)=\sqrt{2t\sigma^2/(1-t)}$ (Lee et al., 9 Aug 2025).

The mel-spectrogram formulation instead writes flow matching directly between noisy and clean speech distributions. Let $s$0 denote the noisy-speech distribution and $s$1 the clean-speech distribution. With linear interpolant

$s$2

the ground-truth velocity is

$s$3

Conditioned on noisy input $s$4 and optional transcript $s$5, the estimator is

$s$6

and the training objective is

$s$7

This formulation adds an auxiliary $s$8 mel-spectrogram reconstruction term, yielding

$s$9

with $y=s+n$0 (Wang et al., 26 May 2025).

Both formulations use MSE supervision on velocity fields along analytically specified paths. This suggests that FlowSE methods derive much of their efficiency from deliberately choosing paths whose transport geometry is easier to integrate than reverse-SDE denoising trajectories.

3. Architectures and conditioning mechanisms

The conditional-CNF FlowSE adopts the same U-Net-style architecture as the BBED score model. The reported components are a time-embedding layer, a convolutional encoder-decoder based on NCSN++, conditioning on $y=s+n$1 via FiLM layers or by concatenating the noisy-speech STFT magnitude as an extra channel, and complex-valued STFT frames $y=s+n$2 that are amplitude compressed and processed as real 2-channel inputs. Total model size, feature-map widths, depth, and hyperparameters are stated to match Lay et al. for fair comparison (Lee et al., 9 Aug 2025).

The mel-spectrogram FlowSE is explicitly modular. It contains a mel-spectrogram encoder $y=s+n$3, a flow-matching generative module $y=s+n$4, and a neural vocoder $y=s+n$5. Given waveform $y=s+n$6, the model computes

$y=s+n$7

The core velocity model is a latent Diffusion Transformer operating on log-mel frames with $y=s+n$8 transformer blocks, 16 attention heads, hidden dimension $y=s+n$9, and feed-forward dimension $$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$0. At each block it injects a sinusoidal time embedding $$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$1 of dimension 1024 and an optional text embedding $$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$2 produced by ConvNeXt V2: $$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$3 Transcripts are tokenized as character sequences, and text is dropped during training with probability $$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$4 to enable text-free operation. The enhanced mel spectrogram is rendered to waveform through a pre-trained neural vocoder, reported as Vocos or BigVGAN in the main FlowSE paper and HiFi-GAN in the later GRPO extension (Wang et al., 26 May 2025, Wang et al., 23 Jan 2026).

The architectural contrast is substantial. One line remains close to diffusion-style score-model backbones for complex STFT features; the other moves toward latent-transformer generation over mel spectrograms with optional linguistic conditioning. The commonality lies in the learned velocity field, not in a canonical network design.

4. Inference, NFEs, and latency

In the conditional-CNF formulation, enhancement is performed by integrating

$$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$5

from $$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$6 to $$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$7. The interval $$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$8 is discretized into $$\frac{dx_t}{dt} = v_t(x_t \mid y), \qquad x_0 \sim \mathcal N(y,\sigma^2 I), \quad t\in[0,1],$$9 time points and integrated with explicit Euler: $x_1 \approx q(s\mid y)$0 Here NFE is simply $x_1 \approx q(s\mid y)$1, the number of calls to $x_1 \approx q(s\mid y)$2. The paper attributes the low-NFE behavior to an optimal-transport-inspired straight-line path in distribution space and states that FlowSE requires as few as $x_1 \approx q(s\mid y)$3 steps, or even fewer, to achieve convergence in distribution, whereas standard diffusion-based SDE solvers typically need 25 to 60 steps (Lee et al., 9 Aug 2025).

In the mel-spectrogram formulation, single-pass sampling starts from Gaussian noise $x_1 \approx q(s\mid y)$4 and integrates

$x_1 \approx q(s\mid y)$5

for $x_1 \approx q(s\mid y)$6 from 1 down to 0 in $x_1 \approx q(s\mid y)$7 steps, with $x_1 \approx q(s\mid y)$8 given as an example. Each step costs one DiT forward pass with complexity $x_1 \approx q(s\mid y)$9, so total cost is approximately $$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$0 one pass. The paper reports that $$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$1 relative to diffusion and therefore yields much lower latency; it also reports that quality saturates by $$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$2, with OVRL drop $$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$3 if $$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$4, while real-time factor halves when $$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$5 (Wang et al., 26 May 2025).

The reported latency measurements make the efficiency claim concrete. On DNS 2021, FlowSE is reported at RTF $$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$6, compared with $$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$7 for CDiffuSE, $$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$8 for SGMSE, and $$L_{\mathrm{CFM}}(\theta)=\mathbb{E}_{t\sim U[0,1-t_\delta],\,(x_1,y)\sim \mathrm{data},\,x_t\sim p_t(\cdot\mid x_1,y)} \left\|v_\theta(x_t,t\mid y)-v_t(x_t\mid x_1,y)\right\|^2 .$$9 for StoRM; the paper summarizes this as $$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$0 faster inference (Wang et al., 26 May 2025). In the conditional-CNF study, reducing NFE from $$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$1 to $$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$2 is described as yielding roughly a $$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$3 speedup in the score/flow model runs, with negligible loss in perceptual quality (Lee et al., 9 Aug 2025).

5. Reported empirical results

The conditional-CNF FlowSE is evaluated on WSJ0-CHiME3 / VB-DMD. At NFE $$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$4, the BBED diffusion model is reported at PESQ $$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$5 and SI-SDR $$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$6. At NFE $$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$7, BBED without fine-tuning drops to PESQ $$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$8 and SI-SDR $$\mu_t(x_1,y)=t\,x_1+(1-t)\,y,\qquad \sigma_t=(1-t)\sigma,$$9, while CRP reaches PESQ $$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$0 and SI-SDR $$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$1. FlowSE at NFE $$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$2 is reported as matching or slightly exceeding CRP in most metrics, with PESQ $$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$3, SI-SDR $$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$4, and DNSMOS, ESTOI, and related measures described as comparable; the paper also states that 5-step FlowSE is approximately equivalent to CRP(5) and BBED(60) (Lee et al., 9 Aug 2025).

The mel-spectrogram FlowSE is evaluated on DNS Challenge 2021 and a simulated VCTK+WHAM!/DEMAND setup. On DNS 2021, the reported OVRL / speaker-similarity / RTF values are $$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$5 without text and $$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$6 with text. For comparison, MaskSR is reported at OVRL $$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$7, speaker similarity $$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$8, RTF $$v_t(x_t\mid x_1,y)= -\frac{\sigma}{(1-t)\sigma}\Bigl(x_t-(t\,x_1+(1-t)\,y)\Bigr)+(x_1-y).$$9; TF-GridNet at OVRL $f(x,y,t)=(y-x)/(1-t)$0, speaker similarity $f(x,y,t)=(y-x)/(1-t)$1, RTF $f(x,y,t)=(y-x)/(1-t)$2; and diffusion baselines between OVRL $f(x,y,t)=(y-x)/(1-t)$3 and $f(x,y,t)=(y-x)/(1-t)$4 with RTF near $f(x,y,t)=(y-x)/(1-t)$5. On the simulated set, FlowSE is reported at WER $f(x,y,t)=(y-x)/(1-t)$6 without text and $f(x,y,t)=(y-x)/(1-t)$7 with text, both at RTF $f(x,y,t)=(y-x)/(1-t)$8, compared with $f(x,y,t)=(y-x)/(1-t)$9 for CDiffuSE, $g(t)=\sqrt{2t\sigma^2/(1-t)}$0 for SGMSE, and $g(t)=\sqrt{2t\sigma^2/(1-t)}$1 for StoRM (Wang et al., 26 May 2025).

The same paper reports that text conditioning gives a small but measurable gain, moving OVRL from $g(t)=\sqrt{2t\sigma^2/(1-t)}$2 to $g(t)=\sqrt{2t\sigma^2/(1-t)}$3 and speaker similarity from $g(t)=\sqrt{2t\sigma^2/(1-t)}$4 to $g(t)=\sqrt{2t\sigma^2/(1-t)}$5. It also reports that a DiT-Large model with $g(t)=\sqrt{2t\sigma^2/(1-t)}$6 improves OVRL by $g(t)=\sqrt{2t\sigma^2/(1-t)}$7 relative to DiT-Small at $g(t)=\sqrt{2t\sigma^2/(1-t)}$8 compute, and that diminishing returns appear beyond $g(t)=\sqrt{2t\sigma^2/(1-t)}$9 integration steps (Wang et al., 26 May 2025).

A later post-training study reports that online GRPO can further improve a base FlowSE model. On DNS2020 No-Reverb, the paper reports SIG $s$00, BAK $s$01, OVRL $s$02, SPK $s$03, and SBS $s$04. On the With-Reverb test, it reports SIG $s$05, BAK $s$06, OVRL $s$07, SPK $s$08, and SBS $s$09. On a real-recording test set, OVRL is reported as $s$10 (Wang et al., 23 Jan 2026).

6. Extensions, misconceptions, and open questions

FlowSE-GRPO extends a pretrained flow-matching SE model with online Group Relative Policy Optimization. The paper formulates an MDP with state $s$11, action $s$12, deterministic transitions, and final reward $s$13. Because GRPO requires stochastic rollouts, it replaces part of the deterministic ODE sampler with an equivalent SDE sampler,

$s$14

with $s$15, and restricts stochastic training to a window $s$16 to reduce compute. A central empirical conclusion is that single-metric optimization rapidly increases the chosen reward but often induces reward hacking that degrades audio fidelity despite higher scores. The proposed mitigation is a multi-metric reward combining DNSMOS, speaker similarity, and SpeechBERTScore, normalized by empirical standard deviation and weighted with $s$17, $s$18 (Wang et al., 23 Jan 2026).

A separate line of work, "Time-Unconditional Generative Speech Enhancement via Autonomous Rectified Flow," challenges an assumption implicit in most FlowSE variants: that explicit time-step conditioning is necessary. Under the linear interpolation

$s$19

the target vector field is

$s$20

which the paper argues is time-invariant. It therefore removes all time-step or noise-scale embeddings from an NCSN++-style U-Net and performs inference with repeated evaluations of the same time-unconditional field. Reported comparisons against FlowSE include PESQ $s$21 vs. $s$22 at NFE $s$23, and PESQ $s$24 vs. $s$25 at NFE $s$26, with RTF $s$27 at NFE $s$28 and $s$29 at NFE $s$30 (Zhang et al., 18 Jun 2026).

This time-conditioning debate is the main conceptual controversy surrounding the FlowSE design space. One position emphasizes explicit time-dependent flows, conditional embeddings, and carefully designed probability paths; the other argues that, for a linear interpolation path, temporal conditioning is redundant. The available evidence supports neither a universal equivalence nor a universal superiority claim across all datasets and feature spaces. What it does establish is that speech enhancement by flow transport remains an active methodological frontier, with open directions explicitly identified as adaptive step-size ODE solvers, better coupling functions, distillation or model compression for on-device real-time SE, extension to multi-channel or reverberant scenarios, joint learning of flow and dispersion $s$31, adaptive $s$32 or non-Gaussian perturbations, and combination with universal SE backbones (Lee et al., 9 Aug 2025, Zhang et al., 18 Jun 2026).