Frequency-Aware Flow-Matching Loss

• Frequency-aware flow-matching loss is a family of training objectives for generative models that explicitly controls distinct frequency components to mitigate spectral bias.
• It extends traditional flow-matching objectives by applying spectral reweighting and amplitude-phase decomposition to enhance high-frequency fidelity in domains like turbulence modeling and time series forecasting.
• Implementations such as FourierFlow and FreqFlow employ dual-branch architectures and efficient frequency-domain computations to achieve superior performance and robust generative capabilities.

Frequency-aware flow-matching loss is a family of training objectives for generative models that enforce the learning of transport velocities (or score fields) in such a way as to explicitly or implicitly control the treatment of distinct frequency components in the generated data. This loss class extends the standard flow-matching objective—essential to recent ODE-based generative modeling frameworks—by formulating, weighting, or augmenting the loss in the spectral (Fourier) domain. The primary motivation is to mitigate spectral bias, a phenomenon where models prioritize low-frequency (long-wavelength) features at the expense of high-frequency content, which is critical in applications such as turbulence modeling or long-term time-series forecasting. Modern instantiations appear in frameworks such as FourierFlow (Wang et al., 1 Jun 2025) and FreqFlow (Moghadas et al., 20 Nov 2025), which employ frequency-aware flow-matching losses to achieve superior fidelity in high-frequency structure and robust, fast generative performance.

1. Core Flow-Matching Objective and Its Frequency-Domain Extension

The foundational element of frequency-aware flow-matching loss is the standard continuous-time flow-matching objective. Given a base sample $u_0 \sim p_0$ (often noise) and data sample $u_1 \sim p_{\text{data}}$, the path-wise linear interpolant $u(t) = (1-t)u_0 + t u_1$, $t \in [0,1]$, leads to a target velocity $$v^*(u(t), t) = \frac{\partial}{\partial t} u(t) = u_1 - u_0$$. The model velocity field $v_\theta(u(t), t)$ is trained by minimizing mean-square error: $$\mathcal{L}_{\text{CFM}} = \mathbb{E}_{t, u_0, u_1}\left[ \| v_\theta(u(t), t) - (u_1 - u_0) \|_2^2 \right].$$

By Parseval's identity, this loss has an exact correspondence in the Fourier domain; given the spatial-to-spectral transforms $\widehat{v}_\theta(t, k)$ and $\widehat{v}^*(t, k)$ over frequencies $k$, the loss becomes

$$\mathcal{L}_{\text{CFM}} = \mathbb{E}_{t} \sum_{k} | \widehat{v}_\theta(t, k) - \widehat{v}^*(t, k) |^2.$$

This spectral formulation permits explicit frequency-dependent reweighting: $$\mathcal{L}_{\text{CFM}}^{\text{freq}} = \mathbb{E}_t \sum_k w(k) | \widehat{v}_\theta(t, k) - \widehat{v}^*(t, k) |^2,$$ where $w(k)$ increases with frequency norm to magnify high-wavenumber loss—commonly $w(k) = 1 + \lambda_{\text{freq}} \|k\|^\eta$ (Wang et al., 1 Jun 2025), or comparable in FreqFlow over temporal frequencies (Moghadas et al., 20 Nov 2025).

2. Implementation in Neural Architectures

FourierFlow: Dual-Branch Backbone and Frequency-Mixing

FourierFlow introduces a dual-branch design integrating a Salient Flow Attention (SFA) branch and a Fourier-Mixing (FM) branch:

• The SFA branch emphasizes local-global attention and is tuned by a differential-attention parameter $\lambda_{\text{diff}}$.
• The FM branch processes intermediate layer features $u^\ell$ via learnable frequency-domain operators:

$$(\mathcal{K} \cdot u^\ell)(t, x) = \mathcal{F}^{-1}[ W_\theta^\ell(\xi) \, \mathcal{F}[u^\ell](\xi) ](x),$$

with frequency-dependent weight functions $$W_\theta^\ell(\xi) = (\beta_\theta^\ell + \alpha_\theta^\ell \|\xi\|^\eta) \bar{W}_\theta^{\ell}$$ ($\eta \geq 1$), increasing high-frequency gain (Wang et al., 1 Jun 2025).

An adaptive gating mechanism fuses the branch outputs; a sigmoid-transformed $1 \times 1$ convolution mixes SFA and FM features, delivering the input to a velocity decoder (typically MLP or convolutional): $$u_{\text{fused}} = G \odot u_{\text{SFA}} + (1-G) \odot u_{\text{FM}}.$$

FreqFlow: Complex Linear Frequency Domain Head

FreqFlow defines a lightweight (89k–140k parameter) architecture. Residual time-series data are mapped to the frequency domain via rFFT, generating complex-valued bins for each channel and example. A deterministic linear flow head $$u_\theta: \mathbb{C}^F \times [0,1] \to \mathbb{C}^F$$ predicts per-frequency velocities, enabling efficient ODE integration in the spectral domain (Moghadas et al., 20 Nov 2025).

3. Spectral Weighting, Amplitude-Phase Decomposition, and Loss Variants

Both FourierFlow and FreqFlow provide mechanisms for reweighting or interpreting loss in the spectral space.

• Explicit weighting compensates for spectral bias—characterized by models' improved performance at low frequencies—and is implemented as $w(k)$ or $w(f)$, where $f$ indexes frequency.
• In FreqFlow, the complex squared error for each bin $f$ is decomposed:

$$\| \hat{u} - u^* \|^2 = (A - A^*)^2 + 2 A A^* [1 - \cos(\phi - \phi^*)],$$

with $A, \phi$ (amplitude, phase) permitting separation into amplitude and phase error terms. This enables practitioners to construct

$$\mathcal{L}_{\text{flow}} = \lambda_{\text{amp}} \mathcal{L}_{\text{amp}} + \lambda_{\text{phase}} \mathcal{L}_{\text{phase}},$$

where $\mathcal{L}_{\text{amp}}$ and $\mathcal{L}_{\text{phase}}$ are frequency-summed amplitude and phase losses (Moghadas et al., 20 Nov 2025).

4. Auxiliary Losses, Regularization, and Residual Modeling

In addition to the primary frequency-aware flow-matching loss, models use auxiliary losses to enhance high-frequency fidelity:

• FourierFlow incorporates a surrogate alignment loss by matching model intermediate layer features to a frozen masked auto-encoder (MAE) trained for high-frequency detail recovery. The alignment loss sums $\ell_2$ feature differences at selected layers:

$$\mathcal{L}_{\text{align}} = \sum_{\ell \in \mathcal{L}} \mathbb{E}_{u_1} \| z_\theta^\ell(u_1) - z_{\text{MAE}}^\ell(u_1) \|_2^2.$$

The total loss is $$\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{CFM}} + \gamma \mathcal{L}_{\text{align}}$$, with alignment weight $\gamma \approx 0.01$ (Wang et al., 1 Jun 2025).

• FreqFlow applies flow-matching only to residual components of inputs (post trend/seasonality removal via moving average or learned interpolation), focusing spectral learning capacity on unpredictable, high-frequency structure essential for accurate long-term forecasts (Moghadas et al., 20 Nov 2025).

5. Training Procedure and Computational Considerations

Training proceeds by sampling base and data samples, interpolating at random $t \sim \text{Uniform}[0,1]$, transforming to the frequency domain as appropriate, and evaluating the loss and auxiliary terms. For FourierFlow, training is performed with AdamW, learning rate $1 \times 10^{-4}$ with cosine decay, batch size 360, and ~200k iterations (Wang et al., 1 Jun 2025). FreqFlow operates with a batch size of 32, learning rate $1 \times 10^{-3}$, and flow-head depth 2–16, achieving end-to-end steps at $\mathcal{O}(B n \log n)$ computational cost due to efficient rFFT usage (Moghadas et al., 20 Nov 2025).

Both frameworks employ standard gradient backpropagation to update network parameters. FourierFlow propagates loss gradients through the dual-branch backbone; FreqFlow supports standard and adjoint-based ODE backpropagation for memory efficiency, though the shallow parameter count allows for straightforward implementation.

6. Theoretical Motivation: Spectral Bias and Signal Recovery

The impetus for frequency-aware losses arises from diffusion and ODE-based generative models' tendency to recover low frequencies before high, particularly under isotropic or homogeneous noise assumptions. Theoretical analyses show the signal-to-noise ratio in each mode $\omega$ scales as $| \hat{x}_0(\omega) |^2 / \int_0^t g(s)^2 \, ds$, and higher-$|\omega|$ modes fall below SNR thresholds earlier in the diffusion process. This formalizes the empirical observation that "Diffusion models reconstruct low frequencies first and high frequencies last," justifying explicit loss reweighting and auxiliary feature alignment to compensate for spectral bias (Wang et al., 1 Jun 2025). FreqFlow's confinement of flow-matching to the residual signals (high-frequency content) emerges as a practical solution to the same challenge (Moghadas et al., 20 Nov 2025).

7. Applications, Performance, and Model Characteristics

Frequency-aware flow-matching objectives underpin generative models for challenging domains where hierarchical, multi-scale, or high-frequency content is critical:

• FourierFlow realizes state-of-the-art results on canonical turbulent flow scenarios, outperforming baseline and advanced diffusion models in out-of-distribution, extrapolation, and noisy input regimes (Wang et al., 1 Jun 2025).
• FreqFlow achieves 7% RMSE improvement over prior methods on long-term multivariate time series forecasting while operating an order of magnitude faster and with fewer parameters—fewer than 140k (Moghadas et al., 20 Nov 2025).

Table: Summary of Frequency-Aware Flow-Matching Loss Properties

Framework	Frequency Domain Usage	Loss Formulation	Auxiliary Regularization
FourierFlow	Spatial (PDE turbulence)	Weighted spectral MSE (implicit via FM branch)	MAE-based feature alignment
FreqFlow	Temporal (MTS forecasting)	Complex spectral MSE, amplitude-phase decomposition	Trend/seasonal removal

A plausible implication is that continued refinement of frequency-domain architectural bias and loss design will advance generative model performance, particularly in settings dominated by multi-scale, non-stationary, or turbulence-like phenomena.