AD-FlowTSE: Adaptive Deterministic Flow Matching

• The paper introduces a paradigm that fuses generative flow matching with adaptive, mixing-ratio-aware trajectory design for target speaker extraction.
• It employs a deterministic ODE-based velocity network with adaptive step-size control to achieve high-fidelity speech extraction in as few as one iteration.
• Empirical results on Libri2Mix demonstrate significant gains in PESQ, ESTOI, and SI-SDR, underscoring the critical role of accurate mixing ratio estimation.

Adaptive Deterministic Flow Matching for Target Speaker Extraction (AD-FlowTSE) is a paradigm in the field of speech separation where flow matching principles from generative modeling are fused with adaptive, data-driven trajectory design to extract the voice of a target speaker from a mixed audio recording. AD-FlowTSE leverages deterministic, mixing-ratio-aware flow trajectories and step-size adaptation, providing efficient, high-fidelity target speech extraction with minimal iterative refinement. The approach defines the transport path in alignment with the mixture composition, enabling direct correction only where interference remains, which can result in strong target speech enhancement with as little as a single integration step (Hsieh et al., 19 Oct 2025, Li et al., 11 Mar 2026).

1. Theoretical Framework and Problem Formulation

AD-FlowTSE is grounded in the generative transport view of target speaker extraction (TSE), where a mixture signal

$x = \tau s_1 + (1{-}\tau) b$

comprises clean target speech $s_1$, background signal $b$ (other speakers, noise), and a mixing ratio $\tau \in [0,1]$ reflecting the energy fraction of the target. The extraction task is represented as transporting $x$ along a learned flow, guided by an external reference (enrollment utterance) $e$, from the mixed domain towards the pure target speech domain.

The core mechanism is an ordinary differential equation: $\frac{dr(t)}{dt} = v_\theta(r(t), t)$ where $v_\theta$ is a neural velocity field that learns to move samples between distributions, parameterized for audio sequences. In deterministic (cold-diffusion) flow matching, the flow is defined explicitly as a linear interpolation: $$x_\tau = (1-\tau) b + \tau s_1, \quad \tau \in [0,1]$$ with the true velocity field

$u_t(x_t \mid b, s_1) = s_1 - b$

(Hsieh et al., 19 Oct 2025).

2. Adaptive Trajectory Design and Mixing-Ratio Awareness

A distinguishing aspect of AD-FlowTSE is the reinterpretation of the time parameter as the mixture’s actual mixing ratio (MR), not as an abstract or uniformly sampled coordinate. The path through signal space is constructed to follow the mixture’s composition, and the initialization point for the flow is placed adaptively at $\tau = \hat\tau$, estimated from the data. This MR-aware initialization ensures that the flow only encompasses the residual interval $[\hat\tau, 1]$, which directly corresponds to the remaining background interference.

The MR predictor $g_\phi(x,e)$—typically based on ECAPA-TDNN speaker embeddings followed by a multilayer perceptron with sigmoid—estimates the mixing ratio by minimizing a supervised mean-squared error loss: $$\mathcal{L}_{\rm MR}(\phi) = \mathbb{E}_{(x,e,\tau)}\left[\left(g_\phi(x,e)-\tau\right)^2\right]$$ At inference, the AD-FlowTSE sampler adapts its step size and number of steps to $1-\hat\tau$, allocating computational resources in proportion to the remaining interference (Hsieh et al., 19 Oct 2025).

3. Velocity Field Learning and Loss Function

The velocity network $v_\theta$ is trained by regressing the predicted vector field against the optimal transport direction for pairs $(b,s_1)$, using the flow-matching loss: $$\mathcal{L}_{\rm FM}(\theta) = \mathbb{E}_{(b,s_1),\,t} \left\lVert v_\theta(x_t, e, t) - (s_1 - b) \right\rVert_2^2$$ where $x_t = (1-t) b + t s_1$ and $e$ is the enrollment. The entire process is fully deterministic: no stochastic noise is added, and the models operate with cold paths (Hsieh et al., 19 Oct 2025, Navon et al., 20 May 2025, Li et al., 11 Mar 2026).

Several AD-FlowTSE variants incorporate more elaborate time-warping or schedule adaptation, such as weighting steps toward the target or utilizing nonuniform reparameterizations. In AlphaFlowTSE (Li et al., 11 Mar 2026), for instance, a JVP-free AlphaFlow objective is employed that anchors the velocity to the data-driven direction and uses interval-consistency with a stop-gradient teacher to maintain coherence on arbitrary finite intervals.

4. Sampling Procedure and Inference Workflow

The AD-FlowTSE inference workflow is precisely defined. Given a test mixture $x$ and enrollment $e$:

1. Estimate $\hat\tau = g_\phi(x, e)$.
2. If $\hat\tau \geq 1-\epsilon$, return $x$ ("very clean": no correction needed).
3. Set the number of steps as $N = \lceil (1-\hat\tau) N_{\max} \rceil$, with step size $\Delta\tau = (1-\hat\tau)/N$.
4. Initialize $x_\tau = x$ (at $\tau = \hat\tau$).
5. For $j = 0\, ...\,N-1$:
   • $t = \hat\tau + j\,\Delta\tau$
   • $$x_\tau \leftarrow x_\tau + \Delta\tau \, v_\theta(x_\tau, e, t)$$
6. Return $x_\tau$ as the extracted target $s_1$.

In practical use, as few as one or five steps (even a single forward call) yield strong performance, with additional iterations sometimes accumulating error rather than improving quality (Hsieh et al., 19 Oct 2025). This enables extremely low-latency, high-efficiency target speech reconstruction.

5. Architectural Considerations

The backbone velocity network in AD-FlowTSE typically consists of a deep transformer-based stack ("diffusion blocks"), with adaptively conditioned normalization and attention mechanisms incorporating time and speaker embedding information (Navon et al., 20 May 2025, Moon et al., 13 Mar 2026, Li et al., 11 Mar 2026). For example, FlowTSE (Navon et al., 20 May 2025) uses 22 transformer layers of width 1024, with adaptive LayerNorm for time conditioning and cross-attention for integrating enrollment features.

Phase reconstruction, when required (e.g., for waveform synthesis), utilizes a specialized vocoder such as a phase-aware extension of Vocos. This vocoder includes cross-attention between mel and STFT features, as well as complex spectral fusion heads, and is trained to jointly maximize signal fidelity and magnitude accuracy (Navon et al., 20 May 2025).

Variant architectures, such as those in AlphaFlowTSE (Li et al., 11 Mar 2026), operate directly in the complex STFT domain and employ UDiT-style (U-Net Diffusion Transformer) backbones, with time and interval length embedded and injected via AdaLN in every block. Teacher-student interval consistency regularization can further stabilize training for reliably one-step inference.

6. Empirical Results and Comparative Evaluation

AD-FlowTSE demonstrates quantitative benefits over baseline discriminative and generative TSE models, as summarized by recent benchmark studies. On Libri2Mix (clean/noisy), for single-step inference with estimated mixing ratio:

• PESQ: 2.89 (clean), 2.15 (noisy)
• ESTOI: 0.90 (clean), 0.81 (noisy)
• SI-SDR: 17.49 dB (clean), 12.69 dB (noisy)
• Speaker similarity (SIM): 0.95 (clean), 0.87 (noisy) (Hsieh et al., 19 Oct 2025, Li et al., 11 Mar 2026)

Reliance on oracle or highly accurate MR estimation is critical; random or grossly misestimated MR degrades performance. In zero-shot transfer scenarios without reliable MR predictors, mixture-to-target trajectory variants such as AlphaFlowTSE (Li et al., 11 Mar 2026) achieve improved generalization, outperforming AD-FlowTSE on perceptual and ASR downstream metrics.

A summary table of results reported in (Li et al., 11 Mar 2026):

Method	Clean PESQ	Noisy PESQ	Clean ESTOI	Noisy ESTOI	SI-SDR (clean/noisy)	SpkSim (clean/noisy)
AD-FlowTSE	2.89	2.15	0.90	0.81	17.49 / 12.70	0.95 / 0.87
AlphaFlowTSE	3.27	2.28	0.94	0.85	19.17 / 13.16	0.93 / 0.76

These results support the efficacy of adaptive deterministic flow matching, while indicating that further robustness to MR errors is attainable via architectural and objective innovations.

7. Limitations and Future Directions

The principal limitation of AD-FlowTSE lies in its dependency on accurate mixing ratio estimation. Severe errors in the MR predictor can lead to under- or over-correction, reducing both SI-SDR and perceptual quality (Hsieh et al., 19 Oct 2025). The approach, as currently formulated, is tailored primarily to additivity-based degradations; extension to more complex acoustic settings (e.g., reverberation, nonlinear interference) may require richer path geometries or injected noise. Future work may involve integrating MR with SNR alignment, exploiting differentiable solvers for adaptive step-size control, or combining AD-FlowTSE with MR-free mixture-to-target objectives for enhanced robustness (Hsieh et al., 19 Oct 2025, Li et al., 11 Mar 2026).