FALCON: Few-step Accurate Likelihoods for CNFs

• The paper introduces FALCON, which reduces the number of neural function evaluations by one to two orders of magnitude while ensuring accurate likelihood estimation.
• It employs flow matching and distillation-based training with invertibility regularization to achieve efficient, tractable sample generation and density computation.
• FALCON's approach benefits applications like molecular Boltzmann sampling and image generative modeling by drastically lowering computational overhead.

Few-step Accurate Likelihoods for Continuous Flows (FALCON) designates a family of flow-based generative modeling techniques that enable both efficient sampling and accurate likelihood evaluation in continuous-time normalizing flows (CNFs) with only a small number of model evaluations per sample or likelihood computation. FALCON is primarily motivated by applications in scientific domains requiring independent samples from intractable target distributions with accurate per-sample likelihoods—such as molecular Boltzmann generators—where previous CNF-based approaches are rendered impractical by the computational cost of integrating high-dimensional ODEs for likelihoods. Recent incarnations of FALCON combine fast flow-matching or distillation-based training objectives with architectural and regularization innovations that preserve invertibility, enabling rapid and scalable likelihood-based inference with minimal discretization overhead (Rehman et al., 10 Dec 2025, Ai et al., 2 Dec 2025).

1. Motivations and Problem Context

In high-dimensional statistical physics, chemistry, and related sciences, a central task is to obtain i.i.d. samples from Boltzmann distributions $p_{\mathrm{target}}(x) \propto \exp(-E(x))$ over coordinate spaces of substantial dimension. While Markov Chain Monte Carlo (MCMC) and molecular dynamics are fundamentally limited by slow mixing in complex energy landscapes, Boltzmann Generators leverage generative models that can propose nearly independent samples and then correct for distribution mismatch via self-normalized importance sampling (SNIS). SNIS critically requires accurate density estimation $p_\theta(x)$ for each generated sample $x$, a bottleneck since standard CNF-based likelihoods require hundreds to thousands of ODE steps per sample for credible density estimates (Rehman et al., 10 Dec 2025). Traditional discretized flows are faster but substantially less accurate.

FALCON addresses this challenge by introducing strategies that reduce the number of required neural function evaluations (NFEs) in both sample generation and likelihood computation by one to two orders of magnitude, without sacrificing accuracy required for downstream SNIS or scientific applications. The framework is also applicable to probabilistic image modeling and reinforcement learning, where efficient model comparison or likelihood-based fine-tuning were previously hindered by the cost of CNF likelihoods (Ai et al., 2 Dec 2025).

2. Mathematical Foundations of Continuous Flows and Likelihoods

Continuous normalizing flows (CNFs) parameterize a transformation from simple base distributions $p_0$ to complex targets $p_1$ via coupled ODEs:

• Flow ODE: $\frac{d}{dt} x(t) = v(x(t), t)\,,\ x(0) \sim p_0$
• Log-likelihood ODE: $$\frac{d}{dt} \log p_t(x(t)) = -\nabla_x \cdot v(x(t), t)$$

The solution defines both a sample $x(1) \sim p_1$ and its exact log-likelihood:

$$\log p_1(x(1)) = \log p_0(x(0)) - \int_0^1 \nabla_x \cdot v(x(t), t)\, dt$$

Inference in CNFs thus involves numerically integrating both the sample trajectory and the associated divergence term—each step incurring significant costs due to trace estimation and the need for fine discretization, especially for high-dimensional $x$ (Ai et al., 2 Dec 2025, Rehman et al., 10 Dec 2025).

3. FALCON Approach: Flow Map Parameterization and Joint Objectives

FALCON replaces the long-trajectory ODE integration of traditional CNFs with a few-step approximation parameterized as an explicit flow map:

$p_\theta(x)$0

where $p_\theta(x)$1 is a neural network trained to simultaneously satisfy:

• Flow matching: At $p_\theta(x)$2, $p_\theta(x)$3, matching the instantaneous velocity.
• Regression toward mean flow: For intervals $p_\theta(x)$4, $p_\theta(x)$5 approximates the trajectory mean or integrated velocity.
• Invertibility regularization: The loss

$p_\theta(x)$6

encourages the map $p_\theta(x)$7 to be invertible, permitting exact change-of-variables and log-density computation.

The overall training objective is then

$p_\theta(x)$8

where $p_\theta(x)$9 is the flow-matching loss, $x$0 is the average (mean flow) regression loss, and $x$1 enforces invertibility (Rehman et al., 10 Dec 2025). In related frameworks such as F2D2 (Ai et al., 2 Dec 2025), a jointly distilled student network incorporates two heads for velocity and divergence, with additional cumulative divergence matching terms.

4. Inference and Likelihood Computation in Few Steps

Once the velocity map $x$2 is trained, inference proceeds via a discretized stepping schedule $x$3. At each step:

$x$4

The cumulative log-density $x$5 approximates $x$6, required for SNIS. Each step is an explicit affine update with a tractable Jacobian determinant, avoiding ODE solvers and trace estimators—enabling inference in as few as 2–16 steps where traditional CNFs would require 100–1000+ (Rehman et al., 10 Dec 2025, Ai et al., 2 Dec 2025). Existing few-step methods can also be adapted by appending a divergence head for likelihood tracking (Ai et al., 2 Dec 2025).

For image generative modeling and related tasks, pseudocode variants—Euler or RK2 stepping—are available and align tightly with the above map structure.

5. Empirical Performance and Applications

In molecular Boltzmann sampling on alanine peptides (ALDP, AL3, AL4, AL6), FALCON achieves effective sample sizes (ESS) and Wasserstein errors (E-W₂, T-W₂) competitive with CNF baselines (ECNF, ECNF++) that require two orders of magnitude more function evaluations. For example, on ALDP:

Model	Steps (NFEs)	ESS	E-W₂	T-W₂
ECNF++	300	0.275	0.914	0.189
FALCON	4	0.067	0.225	0.402

Learning curves demonstrate that FALCON reaches equivalent error thresholds almost $x$7 faster than comparable CNFs. For generative modeling benchmarks such as CIFAR-10 and ImageNet 64×64, a 2–8-step FALCON variant matches or even outperforms long-trajectory CNFs on negative log-likelihood (NLL) and Fréchet Inception Distance (FID) metrics (Ai et al., 2 Dec 2025). Self-guidance techniques, involving a single backward optimization step on the noise initialization, can further boost sample quality—a 2-step MeanFlow-F2D2 model achieves FID lower than a 1024-step teacher.

6. Theoretical Guarantees and Limitations

FALCON’s theoretical foundation rests on two primary results (Rehman et al., 10 Dec 2025):

• If the learned velocity field recovers the continuous-time mean flow, the map $x$8 is globally invertible (by Picard–Lindelöf) and the change-of-variables formula holds exactly.
• If the invertibility regularizer $x$9 is minimized, $p_0$0 satisfies $p_0$1, guaranteeing pointwise invertibility and a valid density estimate.

No finite-step uniform error bounds are provided; empirical likelihood accuracy under a few-step discretization has been validated for SNIS, but rigorous guarantees for SNIS weights or downstream sampling are not proven. There remains a tradeoff, governed by the regularization weight $p_0$2, between perfect invertibility (improving density computation) and sample fidelity (improving generative quality). Current FALCON architectures are not one-step generative, and very low step counts may pose invertibility challenges.

7. Implementation and Extensions

FALCON implementations typically use a U-Net backbone or similar neural architectures, with velocity and optional divergence heads, depending on the chosen training objective. For divergence estimation, a Hutchinson trace estimator is employed where necessary. Training schedules involve pretraining on long-trajectory CNFs (teacher), then distillation or direct regression with inversion and flow-matching losses. Inference schedules (e.g., EDM, with non-uniform step allocation) can impact performance, particularly for more challenging target distributions.

Extensions and future work highlighted in the original studies include the development of structured-Jacobian architectures for even faster log-determinant computations, improved error control, and broader applicability to Bayesian inference, robotics, and model-based reinforcement learning (Rehman et al., 10 Dec 2025, Ai et al., 2 Dec 2025).

---

References:

• "FALCON: Few-step Accurate Likelihoods for Continuous Flows" (Rehman et al., 10 Dec 2025)
• "Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Models" (Ai et al., 2 Dec 2025)