One-Step Score-Based Density Ratio Estimation

Abstract: Density ratio estimation (DRE) is a useful tool for quantifying discrepancies between probability distributions, but existing approaches often involve a trade-off between estimation quality and computational efficiency. Classical direct DRE methods are usually efficient at inference time, yet their performance can seriously deteriorate when the discrepancy between distributions is large. In contrast, score-based DRE methods often yield more accurate estimates in such settings, but they typically require considerable repeated function evaluations and numerical integration. We propose One-step Score-based Density Ratio Estimation (OS-DRE), a partly analytic and solver-free framework designed to combine these complementary advantages. OS-DRE decomposes the time score into spatial and temporal components, representing the latter with an analytic radial basis function (RBF) frame. This formulation converts the otherwise intractable temporal integral into a closed-form weighted sum, thereby removing the need for numerical solvers and enabling DRE with only one function evaluation. We further analyze approximation conditions for the analytic frame, and establish approximation error bounds for both finitely and infinitely smooth temporal kernels, grounding the framework in existing approximation theory. Experiments across density estimation, continual Kullback-Leibler and mutual information estimation, and near out-of-distribution detection demonstrate that OS-DRE offers a favorable balance between estimation quality and inference efficiency.

• The paper proposes a one-step score-based density ratio estimation that bypasses iterative numerical integration for efficient inference.
• It employs a spatiotemporal decomposition with analytic RBFs, providing explicit error bounds and reliable performance in challenging settings.
• Empirical results demonstrate up to 68× faster inference while maintaining sharp density estimation and robust mutual information evaluation.

One-Step Score-Based Density Ratio Estimation: A Formal Analysis

Introduction

This paper introduces "One-Step Score-Based Density Ratio Estimation" (OS-DRE) (2604.10672), a novel framework in the density ratio estimation (DRE) landscape. OS-DRE targets the tension between estimation quality and inference efficiency that plagues existing DRE methodologies. Classical direct estimators—while inference-efficient—are unreliable in high-discrepancy or low-overlap regimes. In contrast, score-based methods offer superior accuracy but at prohibitive inference costs due to their reliance on repeated function evaluations and numerical integration over interpolating distributions.

The OS-DRE framework eliminates this bottleneck via a spatiotemporal decomposition of the log-density ratio path integral, enabling its computation through a single analytic step with precomputed temporal weights. By constructing a frame system based on analytic radial basis functions (RBFs), OS-DRE unifies core advantages from both direct and score-based approaches and theoretically grounds the resulting estimator with explicit error bounds for finite- and infinitely-smooth kernels.

Theoretical Framework

The method rests on the score-based rewriting of the log-density ratio,

$$\log r(x) = \log \frac{p_1(x)}{p_0(x)} = \int_0^1 \partial_t \log p_t(x) \, dt,$$

where $\{p_t\}_{t\in [0,1]}$ defines an interpolating path between $p_0$ and $p_1$. While numerical solvers (e.g., ODE/ODE-based quadrature) are traditionally required, OS-DRE reframes the integral in a manner that entirely bypasses this need.

A key insight is to express the time-score field as a spatiotemporal expansion: $$\partial_t \log p_t(x) \approx \sum_{k=1}^K h_k(x) g_k(t),$$ where $g_k$ are analytic temporal atoms (RBFs) and $h_k(x)$ are learned spatial coefficients. Integrating over time, the log-density ratio becomes

$\log r(x) \approx \sum_{k=1}^K h_k(x) \bar{g}_k,$

with $\bar{g}_k = \int_0^1 g_k(t)\,dt$ as precomputable weights.

The use of non-orthogonal, analytic frames resolves the degeneracy that arises with orthogonal bases (e.g., Fourier, polynomial), where most temporal integrals collapse, discarding useful frequency information. Theoretical development is rigorous, including:

• Justification that the time-score field resides in a suitable Hilbert space, allowing frame-based expansions.
• Explicit construction of analytic RBF families (e.g., Gaussian, IMQ, rational quadratic, Matérn) with closed-form integrals and derivatives for both infinite and finite smoothness regimes.
• Error bounds for finite-term approximations, demonstrating algebraic (Sobolev) or exponential (infinitely smooth kernel) decay as $K$ increases, thereby quantifying the quality-efficiency trade-off.

Practical Implementation

The spatial coefficients are parameterized by neural networks. For a given input, a single network evaluation yields the vector $\{p_t\}_{t\in [0,1]}$0, with the final ratio estimate computed as a dot product against the precomputed vector of temporal integrals: $\{p_t\}_{t\in [0,1]}$1 This formulation yields inference with $\{p_t\}_{t\in [0,1]}$2, independent of the interpolating path complexity.

Training leverages score matching losses, with the analytic form of both integrals and time derivatives enabling first-order optimization and eliminating the need for higher-order automatic differentiation. Empirical studies utilize PyTorch-based models and standard datasets.

Empirical Analysis

Density Estimation and Out-of-Distribution (OOD) Detection

The effectiveness of OS-DRE is validated on synthetic and real-world tabular datasets for density estimation tasks. OS-DRE matches or outperforms computationally-intensive baselines while reducing inference time by up to 68×. The density estimates maintain sharpness and topological correctness even under multimodal, disconnected, or manifold-structured distributions.

A trade-off curve between negative log-likelihood (NLL) and number of function evaluations (NFE) illustrates the quality-efficiency frontier:

Figure 2: NLL versus NFE for density estimation tasks on five tabular datasets, demonstrating OS-DRE's superiority at minimal inference cost.

In OOD detection (CIFAR-100, Near/Far OOD), OS-DRE exhibits clearer separation between ID and OOD score distributions than both neural direct and solver-based score methods:

Figure 4: Densities of OOD scores for ID CIFAR-100 vs. Near-/Far-OOD datasets; OS-DRE and D³RE demonstrate enhanced separation.

Mutual Information (MI) Estimation

Robustness is further established in MI estimation on geometrically pathological benchmarks, where OS-DRE achieves systematically lower mean squared error compared to quadrature-reliant score-based baselines:

Figure 6: MI estimation error (MSE $\{p_t\}_{t\in [0,1]}$3) on four pathological distributions, indicating OS-DRE's efficiency and resilience against complex geometry.

The method also maintains strong sample complexity and inference performance in high-discrepancy scenarios, confirming mitigation of the density-chasm problem even when MI values exceed 20 nats—settings where direct DRE approaches are known to fail.

Kernel and Basis Ablation

Ablative studies explore the effect of basis cardinality ($\{p_t\}_{t\in [0,1]}$4) and kernel choice. Results indicate improvement up to a saturation point, after which overfitting can reduce generalization. IMQ and rational quadratic kernels provide strong general-purpose performance, while Matérn kernels excel for non-smooth temporal fields.

Theoretical and Practical Implications

OS-DRE closes a major methodological gap by enabling score-based DRE with computational efficiency rivaling direct one-pass estimators but without their catastrophic breakdown under sharp, high-dimensional, or non-overlapping regime discrepancies. The closed-form, analytic temporal integration obviates quadrature bias and provides deterministic, stable estimators.

The theoretical framework—embedding spatiotemporal fields into a separable Hilbert space and leveraging frame-based expansions—suggests broader applicability to other path-integral inference problems where the integrand can be learned and admits analytic basis decompositions. The explicit analytic tractability facilitated by RBFs removes a general barrier for practical deployment of score-based methods in real-time or resource-constrained settings.

Conclusion

"One-Step Score-Based Density Ratio Estimation" rigorously establishes a solver-free, analytic pipeline for reliable DRE across diverse modern inference scenarios. By separating the time score into learnable spatial coefficients and analytic temporal frames, OS-DRE provides a practical, theoretically-grounded estimator that is both precise under challenging distributional mismatch and extremely efficient during inference. This development invites future investigation of analytic frame-based approaches within the broader space of path-integral machine learning estimators, including potential extensions to other probabilistic inference and generative modeling tasks.