Papers
Topics
Authors
Recent
Search
2000 character limit reached

Fast Flow Joint Distillation (F2D2)

Updated 5 July 2026
  • The paper introduces F2D2, a joint distillation framework that unifies sample transport and density evolution using a shared velocity field.
  • It reduces computational overhead by converting the continuous ODE system into a few-step flow-map model, cutting neural function evaluations by two orders of magnitude.
  • Empirical results on tasks like CIFAR-10 and ImageNet demonstrate improved likelihood calibration and efficient generation in flow-based models.

Searching arXiv for the cited F2D2 and SCALLOP papers to ground the article in the relevant preprints. Fast Flow Joint Distillation (F2D2) is a framework for jointly accelerating sample generation and likelihood evaluation in continuous normalizing flows and related flow-based generative models. Introduced in “Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Models” (Ai et al., 2 Dec 2025), F2D2 addresses a longstanding asymmetry in generative-model acceleration: prior few-step distillation methods had reduced the neural function evaluations (NFEs) required for sampling, but typically either abandoned tractable likelihoods or still required expensive trajectory integration for density evaluation. F2D2’s central observation is that, in a continuous normalizing flow, the sampling ODE and the likelihood ODE are both determined by a shared velocity field, so both can be distilled into a single few-step flow-map model with a sampling head and a divergence head. The method has subsequently served as the baseline “likelihood flow map” architecture for SCALLOP, which replaces F2D2’s Hutchinson-based divergence supervision with a Hutchinson-free objective (OuYang et al., 27 Jun 2026).

1. Problem setting and motivation

F2D2 targets the computational bottleneck arising when a model must be both fast to sample from and fast to score. In diffusion and flow-based generative modeling, high-quality generation often requires hundreds or thousands of NFEs, and likelihood evaluation is similarly expensive because it typically requires numerical integration of an ODE together with repeated divergence computation along the trajectory (Ai et al., 2 Dec 2025). Even when exact or principled likelihood evaluation is available in theory, the operational cost remains substantial.

This difficulty is especially acute in continuous-time models. For continuous normalizing flows and probability flow ODEs, the state trajectory and the log-density evolve as a coupled system, so likelihood estimation is not a separate post hoc calculation but an additional dynamical quantity that must be propagated with the sample. Existing few-step distillation approaches such as consistency models, shortcut models, and MeanFlow had primarily focused on accelerating the sampling trajectory; they generally did not preserve a directly usable few-step likelihood computation, or they still required expensive integration over full trajectories (Ai et al., 2 Dec 2025).

The practical significance of this gap is explicit in the F2D2 formulation. Log-likelihood evaluation enables model comparison, certain fine-tuning objectives, model selection, and downstream optimization settings where log-likelihoods are useful signals, including PPO/DPO/GRPO-style optimization (Ai et al., 2 Dec 2025). A plausible implication is that F2D2 is best understood not merely as a sampling accelerator, but as an attempt to restore density-aware functionality to the few-step regime.

2. Coupled dynamics and the joint distillation principle

The key theoretical insight of F2D2 is that sampling and likelihood evaluation in a continuous normalizing flow are coupled through the same underlying velocity field vθ(x,t)v_\theta(x,t) (Ai et al., 2 Dec 2025). The sampling dynamics are written as

dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.

Likelihood evolution is governed by the divergence of that field: ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)). Equivalently, the coupled system can be expressed as

[xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},

with

div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).

Backward integration from data to noise gives

logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.

These equations formalize why exact likelihood evaluation in such models requires both trajectory integration and divergence accumulation (Ai et al., 2 Dec 2025).

F2D2 exploits the fact that both subsystems derive from the same vθv_\theta. Rather than distilling only the sampling map, it jointly distills the sampling trajectory and the cumulative divergence using one model (Ai et al., 2 Dec 2025). This yields a few-step model in which the sample update and the log-density update are predicted together. The abstract of the F2D2 paper describes the result as a framework that “simultaneously reduces the number of NFEs required for both sampling and likelihood evaluation by two orders of magnitude” (Ai et al., 2 Dec 2025).

The same joint perspective is retained in the later SCALLOP work, which characterizes F2D2 as a “likelihood flow map” method capable of generating samples and their densities in a small number of function evaluations (OuYang et al., 27 Jun 2026). This continuity is important: SCALLOP is presented not as a replacement for the flow-map idea, but as a modification of the likelihood-training objective within the F2D2 framework.

3. Flow maps and F2D2 parameterization

F2D2 is formulated in terms of flow maps, namely operators that map a state at time tt directly to time ss: Φ(xt,t,s)=xt+tsv(xτ,τ)dτ=xs.\Phi(x_t,t,s) = x_t + \int_t^s v(x_\tau,\tau)\,d\tau = x_s. The model uses the linearized parameterization

dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.0

where dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.1 predicts the average velocity over dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.2, and in the limit dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.3,

dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.4

This is the sampling component of the few-step flow-map model (Ai et al., 2 Dec 2025).

For the joint state dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.5, with dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.6, F2D2 defines a joint flow map

dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.7

where

dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.8

Here dxtdt=vθ(xt,t),x0p0.\frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0.9 is the flow or sampling head, and ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)).0 is the divergence head predicting the average divergence or cumulative log-density change (Ai et al., 2 Dec 2025).

SCALLOP restates this structure in the language of a likelihood flow map. The operator

ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)).1

is defined by

ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)).2

and the parameterization is written in skip-connection form as

ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)).3

with density component

ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)).4

In this formulation, ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)).5 predicts the sample update and ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)).6 predicts the log-density-change or divergence term (OuYang et al., 27 Jun 2026).

The mathematical validity of such maps is tied to standard flow-map characterizations. A map of the form ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)).7 is valid if it satisfies one of the Lagrangian equation, Eulerian equation, or semigroup property. F2D2 uses analogous joint conditions for the augmented state, including the joint Lagrangian condition

ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)).8

the joint Eulerian condition

ddtlogpt(xt)=div(vθ(xt,t)).\frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)).9

and the joint semigroup condition

[xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},0

These conditions provide the structural basis for distilling both trajectory and likelihood into a single learned operator (Ai et al., 2 Dec 2025).

4. Training objective and learning signals

F2D2 trains the joint model through losses that supervise both the instantaneous dynamics and the flow-map consistency of the sample and likelihood branches. In the generic formulation, the objective is

[xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},1

where [xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},2 is a velocity-matching or tangent-condition loss, [xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},3 is a flow-map consistency loss for the sampling branch, [xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},4 matches instantaneous divergence, and [xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},5 is a flow-map consistency loss for the divergence branch (Ai et al., 2 Dec 2025).

The SCALLOP paper presents a more explicit F2D2 decomposition in terms of three losses: [xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},6 The first component is the flow matching loss

[xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},7

with

[xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},8

The second is divergence matching, whose ideal instantaneous form is

[xt logpt;θ(xt)]=[vθ(xt,t) div(vθ(xt,t))],\begin{bmatrix} x_t \ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix},9

The third is self-distillation, for which the paper describes three variants: Lagrangian self-distillation, Eulerian self-distillation, and Progressive self-distillation (OuYang et al., 27 Jun 2026).

For example, the Lagrangian self-distillation loss is written as

div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).0

where div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).1 denotes stop-gradient (OuYang et al., 27 Jun 2026). In the shortcut instantiation described in the F2D2 paper, semigroup consistency is enforced through midpoint self-consistency, while MeanFlow-based F2D2 instead uses the Eulerian identity and an analogous Eulerian residual for the divergence branch (Ai et al., 2 Dec 2025).

These formulations show that F2D2 is not a single training recipe tied to one backbone. It is a joint distillation framework that can be instantiated on multiple flow-map-based few-step samplers. The F2D2 paper explicitly instantiates it on shortcut models and MeanFlow, and describes the framework as compatible with any flow-map-based CNF distillation method (Ai et al., 2 Dec 2025).

5. Divergence estimation, Hutchinson supervision, and architectural modularity

The principal technical difficulty in F2D2 lies on the likelihood side rather than the sampling side. Exact divergence computation in high dimensions is expensive because it requires the trace of a Jacobian. F2D2 therefore uses Hutchinson’s trace estimator, written as

div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).2

where div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).3 is zero-mean random noise, typically div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).4 (OuYang et al., 27 Jun 2026). The SCALLOP paper gives the corresponding Hutchinson Divergence Matching objective as

div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).5

and states that

div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).6

This establishes Hutchinson supervision as an unbiased route to the divergence-matching optimum, albeit with variance (OuYang et al., 27 Jun 2026).

The F2D2 paper emphasizes the practical reason for this choice: exact divergence is too expensive in high dimension, while Hutchinson only requires a Jacobian-vector product. At the same time, both papers note its limitations. The estimator introduces high variance, the optimization becomes noisy, and the Jacobian-vector-product computation carries nontrivial overhead (Ai et al., 2 Dec 2025, OuYang et al., 27 Jun 2026). SCALLOP identifies this variance as one of the main weaknesses of F2D2 (OuYang et al., 27 Jun 2026).

Architecturally, F2D2 is designed to be lightweight. It uses the backbone of an existing few-step flow model and adds one extra scalar divergence head. The two heads share the same backbone, with one head for div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).7 and one head for div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).8; for CIFAR-10 and ImageNet div(vθ(xt,t))=Tr(xtvθ(xt,t)).\operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)).9, the divergence head is implemented as an MLP on the final decoder feature map (Ai et al., 2 Dec 2025). Because the divergence is a scalar, the added capacity is small, and the method is presented as a plug-and-play upgrade rather than a redesign of the underlying flow-map sampler (Ai et al., 2 Dec 2025).

The F2D2 paper also reports practical training procedures, including Hutchinson trace supervision, staged training or warm start, and loss scaling on divergence targets because divergence magnitudes are large and noisy (Ai et al., 2 Dec 2025). The same paper notes limitations: careful early stopping is needed because overtraining can degrade likelihood calibration; ImageNet experiments are limited by compute and use smaller models or fewer iterations than ideal; the study focuses on unconditional generation; performance is sensitive to divergence scaling; and further validation across architectures and modalities is needed (Ai et al., 2 Dec 2025).

6. Inference procedure and empirical behavior

At inference time, the few-step likelihood flow map is applied over a schedule

logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.0

SCALLOP describes inference in the same basic form as F2D2: sample logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.1, initialize logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.2, and for each step logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.3,

logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.4

logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.5

The paper emphasizes that no Jacobian determinant is needed, no exact invertibility is needed, and only a single forward pass of the network per step is required (OuYang et al., 27 Jun 2026). This operational property is central to why F2D2 can score as well as sample in a few-step regime.

The F2D2 experiments are reported on CIFAR-10, ImageNet logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.6, and a 2D checkerboard toy task, using FID on 50k generated images and NLL in bits per dimension on test data (Ai et al., 2 Dec 2025). The teacher flow-matching model on CIFAR-10 attains 3.12 BPD at 1024 steps and FID 2.60 at 200 steps. Selected F2D2 results on CIFAR-10 include:

Model Steps Reported result
Shortcut-Distill-F2D2 8 3.12 BPD, FID 5.68
Shortcut-Distill-F2D2 4 2.87 BPD, FID 5.96
Shortcut-Distill-F2D2 2 2.38 BPD, FID 7.35
Shortcut-Distill-F2D2 1 1.62 BPD, FID 13.76
MeanFlow-F2D2 8 1.91 BPD, FID 3.78
MeanFlow-F2D2 4 1.31 BPD, FID 4.37
MeanFlow-F2D2 2 1.29 BPD, FID 2.59
MeanFlow-F2D2 1 3.33 BPD, FID 3.02

The paper’s interpretation is that, compared with vanilla shortcut and MeanFlow, F2D2 yields much more calibrated likelihoods and in some settings even improves FID (Ai et al., 2 Dec 2025). It further reports 7–171× lower NLL error versus baselines on CIFAR-10 (Ai et al., 2 Dec 2025).

On ImageNet logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.7, the teacher flow-matching model achieves 3.34 BPD at 1024 steps. For Shortcut-Distill-F2D2, the paper reports: 3.51 BPD, FID 21.91 at 8 steps; 3.94 BPD, FID 24.05 at 4 steps; 3.97 BPD, FID 29.83 at 2 steps; and 1.54 BPD, FID 44.02 at 1 step (Ai et al., 2 Dec 2025). The stated conclusion is that F2D2 provides meaningful likelihood estimates close to the teacher while maintaining usable few-step generation, unlike baselines whose likelihoods are invalid (Ai et al., 2 Dec 2025). On the 2D checkerboard task, F2D2 reportedly recovers the true density structure accurately even with 1 NFE, whereas vanilla flow matching and shortcut methods fail badly in few-step likelihood estimation (Ai et al., 2 Dec 2025).

An additional application described in the F2D2 paper is maximum likelihood self-guidance for 2-step MeanFlow. At inference, the method samples logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.8, uses the learned divergence head logp1(x1)=logp0(x0)+10div(vθ(xt,t))dt=logp0(x0)01div(vθ(xt,t))dt.\log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt.9 to estimate a pseudo-negative log-likelihood, takes one Adam step on vθv_\theta0 to reduce that predicted NLL, and then runs the usual few-step sampling. This uses only one forward pass and one backward pass through the likelihood head. The paper claims that 2-step MeanFlow + self-guidance can outperform a 1024-step flow-matching teacher of the same size on CIFAR-10 in FID, with only a single additional backward NFE (Ai et al., 2 Dec 2025). This suggests that fast likelihood estimation may enable inference-time control strategies beyond standard ancestral or deterministic sampling.

7. Relation to SCALLOP and subsequent interpretation

SCALLOP, introduced in “Few-Step Boltzmann Generators via Scalable Likelihood Flow Maps” (OuYang et al., 27 Jun 2026), is explicitly framed as an extension of F2D2. It preserves the idea of the likelihood flow map but replaces the Hutchinson-based divergence loss with a Hutchinson-free conditional divergence matching objective and a vectorized formulation (OuYang et al., 27 Jun 2026). In the SCALLOP analysis, F2D2’s principal limitation is that its density-head training relies on Hutchinson trace estimation, which injects variance, makes optimization noisy, slows training, and can scale poorly in high dimensions (OuYang et al., 27 Jun 2026).

The SCALLOP paper derives a conditional divergence matching loss from the chain rule for log-density along the flow,

vθv_\theta1

and states the conditional divergence matching objective

vθv_\theta2

For the linear interpolation vθv_\theta3, conditioning on vθv_\theta4 renders the conditional density tractable, leading to the stated proposition

vθv_\theta5

which equals vθv_\theta6 up to a constant independent of vθv_\theta7, so that

vθv_\theta8

SCALLOP then introduces vectorized losses targeting vθv_\theta9 rather than the scalar trace, including tt0, tt1, and tt2, and states that

tt3

At optimum,

tt4

with the trace recovered by summing the vector outputs (OuYang et al., 27 Jun 2026).

Empirically, SCALLOP reports that it consistently improves over F2D2 on molecular Boltzmann-generator benchmarks ALA-2, ALA-3, ALA-4, and ALA-6, using ESS, E-W2, and T-W2 as metrics (OuYang et al., 27 Jun 2026). Reported examples include ALA-2 ESS improving from tt5 for F2D2 to tt6 for SCALLOP, and ALA-4 E-W2 improving from tt7 to tt8 (OuYang et al., 27 Jun 2026). The paper further reports up to 100x reduction in training variance, higher gradient signal-to-noise ratio, faster convergence, about 10% training speedup over F2D2 on the same GPU, and nearly identical inference cost because both methods use appended readout heads rather than Jacobian determinants (OuYang et al., 27 Jun 2026). On CelebA-64, SCALLOP is reported to achieve comparable BPD and FID to F2D2 across 1, 2, 4, and 8 Euler steps while consistently reducing per-sample BPD error, with loss computation taking only 83.1% of F2D2’s wall-clock time (OuYang et al., 27 Jun 2026).

In this later perspective, F2D2 occupies a historically important position as the first explicit framework for few-step joint distillation of sampling and likelihood in flow-based models, while SCALLOP is presented as a refinement of its divergence-learning mechanism rather than a departure from its core formulation (OuYang et al., 27 Jun 2026). A common misconception is that F2D2’s main contribution is only a fast sampler with an auxiliary scalar head. The papers instead define it more precisely as a joint flow-map framework that distills both the trajectory and the cumulative divergence into a single few-step model (Ai et al., 2 Dec 2025). Another misconception is that few-step generative distillation necessarily sacrifices usable likelihoods; F2D2 was introduced specifically to challenge that assumption by keeping likelihood computation within the few-step operating regime (Ai et al., 2 Dec 2025).

Taken together, the two papers position F2D2 as a modular likelihood-aware extension of flow-map distillation. Its main contribution is the formal and practical unification of sample transport and density evolution under a shared few-step map, enabling fast scoring as well as fast generation (Ai et al., 2 Dec 2025, OuYang et al., 27 Jun 2026).

Definition Search Book Streamline Icon: https://streamlinehq.com
References (2)

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Fast Flow Joint Distillation (F2D2).