--- title: Fast Flow Joint Distillation (F2D2) url: https://www.emergentmind.com/topics/fast-flow-joint-distillation-f2d2 type: topic --- # Fast Flow Joint Distillation (F2D2) Searching arXiv for the cited F2D2 and SCALLOP papers to ground the article in the relevant preprints. Fast Flow Joint Distillation (F2D2) is a framework for jointly accelerating sample generation and likelihood evaluation in continuous normalizing flows and related flow-based generative models. Introduced in “Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Models” [2512.02636], F2D2 addresses a longstanding asymmetry in generative-model acceleration: prior few-step distillation methods had reduced the neural function evaluations (NFEs) required for sampling, but typically either abandoned tractable likelihoods or still required expensive trajectory integration for density evaluation. F2D2’s central observation is that, in a continuous normalizing flow, the sampling ODE and the likelihood ODE are both determined by a shared velocity field, so both can be distilled into a single few-step flow-map model with a sampling head and a divergence head. The method has subsequently served as the baseline “likelihood flow map” architecture for SCALLOP, which replaces F2D2’s Hutchinson-based divergence supervision with a Hutchinson-free objective [2606.29110]. ## 1. Problem setting and motivation F2D2 targets the computational bottleneck arising when a model must be both fast to sample from and fast to score. In diffusion and flow-based generative modeling, high-quality generation often requires hundreds or thousands of NFEs, and likelihood evaluation is similarly expensive because it typically requires numerical integration of an ODE together with repeated divergence computation along the trajectory [2512.02636]. Even when exact or principled likelihood evaluation is available in theory, the operational cost remains substantial. This difficulty is especially acute in continuous-time models. For continuous normalizing flows and probability flow ODEs, the state trajectory and the log-density evolve as a coupled system, so likelihood estimation is not a separate post hoc calculation but an additional dynamical quantity that must be propagated with the sample. Existing few-step distillation approaches such as consistency models, shortcut models, and MeanFlow had primarily focused on accelerating the sampling trajectory; they generally did not preserve a directly usable few-step likelihood computation, or they still required expensive integration over full trajectories [2512.02636]. The practical significance of this gap is explicit in the F2D2 formulation. Log-likelihood evaluation enables model comparison, certain fine-tuning objectives, model selection, and downstream optimization settings where log-likelihoods are useful signals, including PPO/DPO/GRPO-style optimization [2512.02636]. A plausible implication is that F2D2 is best understood not merely as a sampling accelerator, but as an attempt to restore density-aware functionality to the few-step regime. ## 2. Coupled dynamics and the joint distillation principle The key theoretical insight of F2D2 is that sampling and likelihood evaluation in a continuous normalizing flow are coupled through the same underlying velocity field \(v_\theta(x,t)\) [2512.02636]. The sampling dynamics are written as \[ \frac{d x_t}{dt} = v_\theta(x_t,t), \qquad x_0 \sim p_0. \] Likelihood evolution is governed by the divergence of that field: \[ \frac{d}{dt}\log p_t(x_t) = -\operatorname{div}(v_\theta(x_t,t)). \] Equivalently, the coupled system can be expressed as \[ \begin{bmatrix} x_t \\ \log p_{t;\theta}(x_t) \end{bmatrix} = \begin{bmatrix} v_\theta(x_t,t) \\ -\operatorname{div}(v_\theta(x_t,t)) \end{bmatrix}, \] with \[ \operatorname{div}(v_\theta(x_t,t)) = \operatorname{Tr}(\nabla_{x_t} v_\theta(x_t,t)). \] Backward integration from data to noise gives \[ \log p_{1}(x_1) = \log p_0(x_0) + \int_1^0 \operatorname{div}(v_\theta(x_t,t))\,dt = \log p_0(x_0) - \int_0^1 \operatorname{div}(v_\theta(x_t,t))\,dt. \] These equations formalize why exact likelihood evaluation in such models requires both trajectory integration and divergence accumulation [2512.02636]. F2D2 exploits the fact that both subsystems derive from the same \(v_\theta\). Rather than distilling only the sampling map, it jointly distills the sampling trajectory and the cumulative divergence using one model [2512.02636]. This yields a few-step model in which the sample update and the log-density update are predicted together. The abstract of the F2D2 paper describes the result as a framework that “simultaneously reduces the number of NFEs required for both sampling and likelihood evaluation by two orders of magnitude” [2512.02636]. The same joint perspective is retained in the later SCALLOP work, which characterizes F2D2 as a “likelihood flow map” method capable of generating samples and their densities in a small number of function evaluations [2606.29110]. This continuity is important: SCALLOP is presented not as a replacement for the flow-map idea, but as a modification of the likelihood-training objective within the F2D2 framework. ## 3. Flow maps and F2D2 parameterization F2D2 is formulated in terms of flow maps, namely operators that map a state at time \(t\) directly to time \(s\): \[ \Phi(x_t,t,s) = x_t + \int_t^s v(x_\tau,\tau)\,d\tau = x_s. \] The model uses the linearized parameterization \[ \Phi_\theta(x_t,t,s) = x_t + (s-t)u_\theta(x_t,t,s), \] where \(u_\theta\) predicts the average velocity over \([t,s]\), and in the limit \(s \to t\), \[ u_\theta(x_t,t,t) \approx v_\theta(x_t,t). \] This is the sampling component of the few-step flow-map model [2512.02636]. For the joint state \(y_t = (x_t, z_t)^\top\), with \(z_t = \log p_t(x_t)\), F2D2 defines a joint flow map \[ \Phi_{Y;\theta}(y_t,t,s) = \begin{bmatrix} \Phi_X(x_t,t,s) \\ \Phi_Z(x_t,z_t,t,s) \end{bmatrix} = y_t + (s-t)f_\theta(x_t,t,s), \] where \[ f_\theta(x_t,t,s) = \begin{bmatrix} u_\theta(x_t,t,s) \\ D_\theta(x_t,t,s) \end{bmatrix}. \] Here \(u_\theta\) is the flow or sampling head, and \(D_\theta\) is the divergence head predicting the average divergence or cumulative log-density change [2512.02636]. SCALLOP restates this structure in the language of a likelihood flow map. The operator \[ \Upsilon : \mathbb{R}^d \times [0,1]^2 \to \mathbb{R}^{d+1} \] is defined by \[ \Upsilon(x_t,t,s) = \Big[x_s,\ \log p_s(x_s)-\log p_t(x_t)\Big], \tag{6} \] and the parameterization is written in skip-connection form as \[ \Upsilon_\theta(x_t,t,s) = x_t + (s-t)\,u_\theta(x_t,t,s), \quad u_\theta(x_t,t,t)=v_t(x_t), \] with density component \[ D_\theta(x_t,t,t) = -\nabla\!\cdot u_t(x_t). \tag{7} \] In this formulation, \(u_\theta\) predicts the sample update and \(D_\theta\) predicts the log-density-change or divergence term [2606.29110]. The mathematical validity of such maps is tied to standard flow-map characterizations. A map of the form \(\Phi(x,t,s)=x+(s-t)u(x,t,s)\) is valid if it satisfies one of the Lagrangian equation, Eulerian equation, or semigroup property. F2D2 uses analogous joint conditions for the augmented state, including the joint Lagrangian condition \[ \partial_s \Phi_Y(y,t,s) = f(\Phi_Y(y,t,s),s,s), \] the joint Eulerian condition \[ \partial_t \Phi_Y(y,t,s) + \nabla_y \Phi_Y(y,t,s) f(y,t,t) = 0, \] and the joint semigroup condition \[ \Phi_Y(y,t,s) = \Phi_Y(\Phi_Y(y,t,r),r,s), \qquad t