Fast Flow Joint Distillation (F2D2)
- The paper introduces F2D2, a joint distillation framework that unifies sample transport and density evolution using a shared velocity field.
- It reduces computational overhead by converting the continuous ODE system into a few-step flow-map model, cutting neural function evaluations by two orders of magnitude.
- Empirical results on tasks like CIFAR-10 and ImageNet demonstrate improved likelihood calibration and efficient generation in flow-based models.
Searching arXiv for the cited F2D2 and SCALLOP papers to ground the article in the relevant preprints. Fast Flow Joint Distillation (F2D2) is a framework for jointly accelerating sample generation and likelihood evaluation in continuous normalizing flows and related flow-based generative models. Introduced in “Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Models” (Ai et al., 2 Dec 2025), F2D2 addresses a longstanding asymmetry in generative-model acceleration: prior few-step distillation methods had reduced the neural function evaluations (NFEs) required for sampling, but typically either abandoned tractable likelihoods or still required expensive trajectory integration for density evaluation. F2D2’s central observation is that, in a continuous normalizing flow, the sampling ODE and the likelihood ODE are both determined by a shared velocity field, so both can be distilled into a single few-step flow-map model with a sampling head and a divergence head. The method has subsequently served as the baseline “likelihood flow map” architecture for SCALLOP, which replaces F2D2’s Hutchinson-based divergence supervision with a Hutchinson-free objective (OuYang et al., 27 Jun 2026).
1. Problem setting and motivation
F2D2 targets the computational bottleneck arising when a model must be both fast to sample from and fast to score. In diffusion and flow-based generative modeling, high-quality generation often requires hundreds or thousands of NFEs, and likelihood evaluation is similarly expensive because it typically requires numerical integration of an ODE together with repeated divergence computation along the trajectory (Ai et al., 2 Dec 2025). Even when exact or principled likelihood evaluation is available in theory, the operational cost remains substantial.
This difficulty is especially acute in continuous-time models. For continuous normalizing flows and probability flow ODEs, the state trajectory and the log-density evolve as a coupled system, so likelihood estimation is not a separate post hoc calculation but an additional dynamical quantity that must be propagated with the sample. Existing few-step distillation approaches such as consistency models, shortcut models, and MeanFlow had primarily focused on accelerating the sampling trajectory; they generally did not preserve a directly usable few-step likelihood computation, or they still required expensive integration over full trajectories (Ai et al., 2 Dec 2025).
The practical significance of this gap is explicit in the F2D2 formulation. Log-likelihood evaluation enables model comparison, certain fine-tuning objectives, model selection, and downstream optimization settings where log-likelihoods are useful signals, including PPO/DPO/GRPO-style optimization (Ai et al., 2 Dec 2025). A plausible implication is that F2D2 is best understood not merely as a sampling accelerator, but as an attempt to restore density-aware functionality to the few-step regime.
2. Coupled dynamics and the joint distillation principle
The key theoretical insight of F2D2 is that sampling and likelihood evaluation in a continuous normalizing flow are coupled through the same underlying velocity field (Ai et al., 2 Dec 2025). The sampling dynamics are written as
Likelihood evolution is governed by the divergence of that field: Equivalently, the coupled system can be expressed as
with
Backward integration from data to noise gives
These equations formalize why exact likelihood evaluation in such models requires both trajectory integration and divergence accumulation (Ai et al., 2 Dec 2025).
F2D2 exploits the fact that both subsystems derive from the same . Rather than distilling only the sampling map, it jointly distills the sampling trajectory and the cumulative divergence using one model (Ai et al., 2 Dec 2025). This yields a few-step model in which the sample update and the log-density update are predicted together. The abstract of the F2D2 paper describes the result as a framework that “simultaneously reduces the number of NFEs required for both sampling and likelihood evaluation by two orders of magnitude” (Ai et al., 2 Dec 2025).
The same joint perspective is retained in the later SCALLOP work, which characterizes F2D2 as a “likelihood flow map” method capable of generating samples and their densities in a small number of function evaluations (OuYang et al., 27 Jun 2026). This continuity is important: SCALLOP is presented not as a replacement for the flow-map idea, but as a modification of the likelihood-training objective within the F2D2 framework.
3. Flow maps and F2D2 parameterization
F2D2 is formulated in terms of flow maps, namely operators that map a state at time directly to time : The model uses the linearized parameterization
0
where 1 predicts the average velocity over 2, and in the limit 3,
4
This is the sampling component of the few-step flow-map model (Ai et al., 2 Dec 2025).
For the joint state 5, with 6, F2D2 defines a joint flow map
7
where
8
Here 9 is the flow or sampling head, and 0 is the divergence head predicting the average divergence or cumulative log-density change (Ai et al., 2 Dec 2025).
SCALLOP restates this structure in the language of a likelihood flow map. The operator
1
is defined by
2
and the parameterization is written in skip-connection form as
3
with density component
4
In this formulation, 5 predicts the sample update and 6 predicts the log-density-change or divergence term (OuYang et al., 27 Jun 2026).
The mathematical validity of such maps is tied to standard flow-map characterizations. A map of the form 7 is valid if it satisfies one of the Lagrangian equation, Eulerian equation, or semigroup property. F2D2 uses analogous joint conditions for the augmented state, including the joint Lagrangian condition
8
the joint Eulerian condition
9
and the joint semigroup condition
0
These conditions provide the structural basis for distilling both trajectory and likelihood into a single learned operator (Ai et al., 2 Dec 2025).
4. Training objective and learning signals
F2D2 trains the joint model through losses that supervise both the instantaneous dynamics and the flow-map consistency of the sample and likelihood branches. In the generic formulation, the objective is
1
where 2 is a velocity-matching or tangent-condition loss, 3 is a flow-map consistency loss for the sampling branch, 4 matches instantaneous divergence, and 5 is a flow-map consistency loss for the divergence branch (Ai et al., 2 Dec 2025).
The SCALLOP paper presents a more explicit F2D2 decomposition in terms of three losses: 6 The first component is the flow matching loss
7
with
8
The second is divergence matching, whose ideal instantaneous form is
9
The third is self-distillation, for which the paper describes three variants: Lagrangian self-distillation, Eulerian self-distillation, and Progressive self-distillation (OuYang et al., 27 Jun 2026).
For example, the Lagrangian self-distillation loss is written as
0
where 1 denotes stop-gradient (OuYang et al., 27 Jun 2026). In the shortcut instantiation described in the F2D2 paper, semigroup consistency is enforced through midpoint self-consistency, while MeanFlow-based F2D2 instead uses the Eulerian identity and an analogous Eulerian residual for the divergence branch (Ai et al., 2 Dec 2025).
These formulations show that F2D2 is not a single training recipe tied to one backbone. It is a joint distillation framework that can be instantiated on multiple flow-map-based few-step samplers. The F2D2 paper explicitly instantiates it on shortcut models and MeanFlow, and describes the framework as compatible with any flow-map-based CNF distillation method (Ai et al., 2 Dec 2025).
5. Divergence estimation, Hutchinson supervision, and architectural modularity
The principal technical difficulty in F2D2 lies on the likelihood side rather than the sampling side. Exact divergence computation in high dimensions is expensive because it requires the trace of a Jacobian. F2D2 therefore uses Hutchinson’s trace estimator, written as
2
where 3 is zero-mean random noise, typically 4 (OuYang et al., 27 Jun 2026). The SCALLOP paper gives the corresponding Hutchinson Divergence Matching objective as
5
and states that
6
This establishes Hutchinson supervision as an unbiased route to the divergence-matching optimum, albeit with variance (OuYang et al., 27 Jun 2026).
The F2D2 paper emphasizes the practical reason for this choice: exact divergence is too expensive in high dimension, while Hutchinson only requires a Jacobian-vector product. At the same time, both papers note its limitations. The estimator introduces high variance, the optimization becomes noisy, and the Jacobian-vector-product computation carries nontrivial overhead (Ai et al., 2 Dec 2025, OuYang et al., 27 Jun 2026). SCALLOP identifies this variance as one of the main weaknesses of F2D2 (OuYang et al., 27 Jun 2026).
Architecturally, F2D2 is designed to be lightweight. It uses the backbone of an existing few-step flow model and adds one extra scalar divergence head. The two heads share the same backbone, with one head for 7 and one head for 8; for CIFAR-10 and ImageNet 9, the divergence head is implemented as an MLP on the final decoder feature map (Ai et al., 2 Dec 2025). Because the divergence is a scalar, the added capacity is small, and the method is presented as a plug-and-play upgrade rather than a redesign of the underlying flow-map sampler (Ai et al., 2 Dec 2025).
The F2D2 paper also reports practical training procedures, including Hutchinson trace supervision, staged training or warm start, and loss scaling on divergence targets because divergence magnitudes are large and noisy (Ai et al., 2 Dec 2025). The same paper notes limitations: careful early stopping is needed because overtraining can degrade likelihood calibration; ImageNet experiments are limited by compute and use smaller models or fewer iterations than ideal; the study focuses on unconditional generation; performance is sensitive to divergence scaling; and further validation across architectures and modalities is needed (Ai et al., 2 Dec 2025).
6. Inference procedure and empirical behavior
At inference time, the few-step likelihood flow map is applied over a schedule
0
SCALLOP describes inference in the same basic form as F2D2: sample 1, initialize 2, and for each step 3,
4
5
The paper emphasizes that no Jacobian determinant is needed, no exact invertibility is needed, and only a single forward pass of the network per step is required (OuYang et al., 27 Jun 2026). This operational property is central to why F2D2 can score as well as sample in a few-step regime.
The F2D2 experiments are reported on CIFAR-10, ImageNet 6, and a 2D checkerboard toy task, using FID on 50k generated images and NLL in bits per dimension on test data (Ai et al., 2 Dec 2025). The teacher flow-matching model on CIFAR-10 attains 3.12 BPD at 1024 steps and FID 2.60 at 200 steps. Selected F2D2 results on CIFAR-10 include:
| Model | Steps | Reported result |
|---|---|---|
| Shortcut-Distill-F2D2 | 8 | 3.12 BPD, FID 5.68 |
| Shortcut-Distill-F2D2 | 4 | 2.87 BPD, FID 5.96 |
| Shortcut-Distill-F2D2 | 2 | 2.38 BPD, FID 7.35 |
| Shortcut-Distill-F2D2 | 1 | 1.62 BPD, FID 13.76 |
| MeanFlow-F2D2 | 8 | 1.91 BPD, FID 3.78 |
| MeanFlow-F2D2 | 4 | 1.31 BPD, FID 4.37 |
| MeanFlow-F2D2 | 2 | 1.29 BPD, FID 2.59 |
| MeanFlow-F2D2 | 1 | 3.33 BPD, FID 3.02 |
The paper’s interpretation is that, compared with vanilla shortcut and MeanFlow, F2D2 yields much more calibrated likelihoods and in some settings even improves FID (Ai et al., 2 Dec 2025). It further reports 7–171× lower NLL error versus baselines on CIFAR-10 (Ai et al., 2 Dec 2025).
On ImageNet 7, the teacher flow-matching model achieves 3.34 BPD at 1024 steps. For Shortcut-Distill-F2D2, the paper reports: 3.51 BPD, FID 21.91 at 8 steps; 3.94 BPD, FID 24.05 at 4 steps; 3.97 BPD, FID 29.83 at 2 steps; and 1.54 BPD, FID 44.02 at 1 step (Ai et al., 2 Dec 2025). The stated conclusion is that F2D2 provides meaningful likelihood estimates close to the teacher while maintaining usable few-step generation, unlike baselines whose likelihoods are invalid (Ai et al., 2 Dec 2025). On the 2D checkerboard task, F2D2 reportedly recovers the true density structure accurately even with 1 NFE, whereas vanilla flow matching and shortcut methods fail badly in few-step likelihood estimation (Ai et al., 2 Dec 2025).
An additional application described in the F2D2 paper is maximum likelihood self-guidance for 2-step MeanFlow. At inference, the method samples 8, uses the learned divergence head 9 to estimate a pseudo-negative log-likelihood, takes one Adam step on 0 to reduce that predicted NLL, and then runs the usual few-step sampling. This uses only one forward pass and one backward pass through the likelihood head. The paper claims that 2-step MeanFlow + self-guidance can outperform a 1024-step flow-matching teacher of the same size on CIFAR-10 in FID, with only a single additional backward NFE (Ai et al., 2 Dec 2025). This suggests that fast likelihood estimation may enable inference-time control strategies beyond standard ancestral or deterministic sampling.
7. Relation to SCALLOP and subsequent interpretation
SCALLOP, introduced in “Few-Step Boltzmann Generators via Scalable Likelihood Flow Maps” (OuYang et al., 27 Jun 2026), is explicitly framed as an extension of F2D2. It preserves the idea of the likelihood flow map but replaces the Hutchinson-based divergence loss with a Hutchinson-free conditional divergence matching objective and a vectorized formulation (OuYang et al., 27 Jun 2026). In the SCALLOP analysis, F2D2’s principal limitation is that its density-head training relies on Hutchinson trace estimation, which injects variance, makes optimization noisy, slows training, and can scale poorly in high dimensions (OuYang et al., 27 Jun 2026).
The SCALLOP paper derives a conditional divergence matching loss from the chain rule for log-density along the flow,
1
and states the conditional divergence matching objective
2
For the linear interpolation 3, conditioning on 4 renders the conditional density tractable, leading to the stated proposition
5
which equals 6 up to a constant independent of 7, so that
8
SCALLOP then introduces vectorized losses targeting 9 rather than the scalar trace, including 0, 1, and 2, and states that
3
At optimum,
4
with the trace recovered by summing the vector outputs (OuYang et al., 27 Jun 2026).
Empirically, SCALLOP reports that it consistently improves over F2D2 on molecular Boltzmann-generator benchmarks ALA-2, ALA-3, ALA-4, and ALA-6, using ESS, E-W2, and T-W2 as metrics (OuYang et al., 27 Jun 2026). Reported examples include ALA-2 ESS improving from 5 for F2D2 to 6 for SCALLOP, and ALA-4 E-W2 improving from 7 to 8 (OuYang et al., 27 Jun 2026). The paper further reports up to 100x reduction in training variance, higher gradient signal-to-noise ratio, faster convergence, about 10% training speedup over F2D2 on the same GPU, and nearly identical inference cost because both methods use appended readout heads rather than Jacobian determinants (OuYang et al., 27 Jun 2026). On CelebA-64, SCALLOP is reported to achieve comparable BPD and FID to F2D2 across 1, 2, 4, and 8 Euler steps while consistently reducing per-sample BPD error, with loss computation taking only 83.1% of F2D2’s wall-clock time (OuYang et al., 27 Jun 2026).
In this later perspective, F2D2 occupies a historically important position as the first explicit framework for few-step joint distillation of sampling and likelihood in flow-based models, while SCALLOP is presented as a refinement of its divergence-learning mechanism rather than a departure from its core formulation (OuYang et al., 27 Jun 2026). A common misconception is that F2D2’s main contribution is only a fast sampler with an auxiliary scalar head. The papers instead define it more precisely as a joint flow-map framework that distills both the trajectory and the cumulative divergence into a single few-step model (Ai et al., 2 Dec 2025). Another misconception is that few-step generative distillation necessarily sacrifices usable likelihoods; F2D2 was introduced specifically to challenge that assumption by keeping likelihood computation within the few-step operating regime (Ai et al., 2 Dec 2025).
Taken together, the two papers position F2D2 as a modular likelihood-aware extension of flow-map distillation. Its main contribution is the formal and practical unification of sample transport and density evolution under a shared few-step map, enabling fast scoring as well as fast generation (Ai et al., 2 Dec 2025, OuYang et al., 27 Jun 2026).