Energy Score Models Overview

• Energy score models are generative frameworks that fuse energy-based modeling with score matching methods to bypass computing intractable partition functions.
• They leverage techniques like denoising, sliced, and variational score matching to efficiently synthesize high-dimensional data and enable robust posterior inference.
• Applications span image synthesis, molecular structure refinement, and latent variable modeling while also promoting advanced hybrid sampling strategies.

Energy score models are a class of generative and density modeling frameworks that fuse energy-based modeling (EBMs) with score-based learning objectives, notably score matching and its generalizations. These models leverage the mathematical equivalence between the gradient of the log-density (the "score") and the force fields arising from the underlying energy function, enabling efficient training without evaluation of intractable partition functions. Over the last decade, developments in denoising score matching, sliced score matching, Riemannian extensions, and hybrid score-energy frameworks have expanded the application of energy score models to high-dimensional data synthesis, molecular structure refinement, latent variable modeling, robust posterior inference, and advanced MCMC sampling procedures.

1. Fundamental Concepts and Mathematical Formulation

Energy-based models parameterize unnormalized probability densities as $p_\theta(x) = \exp(-E_\theta(x))/Z_\theta$, where $E_\theta(x)$ is a scalar energy function, and $Z_\theta$ is the partition function. The model’s score function is $$s_\theta(x) = \nabla_x \log p_\theta(x) = -\nabla_x E_\theta(x)$$. Direct maximum likelihood training is rarely practical, given the cost of evaluating or differentiating $Z_\theta$. Score matching, as pioneered by Hyvärinen, circumvents this by minimizing the Fisher divergence:

$$L_{\text{SM}}(\theta) = \frac{1}{2} \mathbb{E}_{p_{\text{data}}} \Big[ \|s_\theta(x) - s_{\text{data}}(x)\|^2 \Big ]$$

Practical implementations reformulate this via integration by parts, eliminating the need to compute $s_{\text{data}}$ explicitly. Denoising score matching interprets scores as force fields that guide optimization on data manifolds (Li et al., 2019), and sliced score matching (SSM) uses random projections to efficiently estimate high-dimensional Hessian-vector products (Song et al., 2019).

Recent extensions include multi-scale denoising score matching (MDSM) to combat the shell concentration effect at high dimension (Li et al., 2019), Riemannian denoising score matching for molecular geometry refinement where scores operate on a physics-informed manifold (Woo et al., 2024), and reparameterizations that connect flow-based generative models to EBMs, yielding computational gains by bypassing Jacobian determinants (Chao et al., 2023).

2. Score Matching Variants and Computational Efficiency

Energy score models are principally trained via the following methodologies:

• Classic score matching requires trace-of-Hessian evaluation, impractical for deep architectures or high $D$.
• Denoising score matching (DSM) leverages data-perturbation with known noise kernels; the objective is equivalent to matching the score of a Gaussian-smoothed density (Li et al., 2019).
• Sliced score matching (SSM) projects scores onto random directions, reducing the cost to two gradient passes per slice and enabling deployment in deep, high-dimensional models. Consistency and asymptotic normality of SSM estimators are formally established (Song et al., 2019).
• Energy discrepancy loss (ED) provides a score-independent alternative by directly comparing energies on perturbed data. ED interpolates between score matching and maximum-likelihood as the noise scale varies, and is convex in the energy function with unique minimizer at the true log-density (Schröder et al., 2023).

These approaches are summarized in the table below:

Objective	Key Features	Required Derivatives
Classic SM	Partition-free	Full Hessian trace
DSM	Noise smoothing	First derivatives
SSM	Projections, scalable	Hessian-vector products
Energy Discrepancy	Score-free, global matching	None (forward only)

3. Structured Extensions: Latent Variable, Manifold, and Conservative Models

Latent variable EBMs

Training energy-based latent variable models presents additional challenges due to intractable posteriors $p_\theta(z|x)$ and marginalization. Solutions include:

• Bi-level Score Matching (BiSM), where a variational posterior $q_\phi(z|x)$ is co-optimized via bi-level stochastic gradient unrolling, yielding unbiased score estimates under mild regularity (Bao et al., 2020).
• Variational score estimation (VaES/VaGES) uses an amortized variational posterior to generate unbiased, differentiable estimators for both score and score gradients, plugging into kernelized Stein discrepancy or Fisher divergence objectives with rigorous bias control (Bao et al., 2020).

Riemannian extensions

In molecular structure refinement, the Riemannian denoising score matching (R-DSM) procedure embeds molecular conformers in a manifold defined by internal coordinates, with the score aligning closely to the true gradient of the potential energy surface. The forward-noising and reverse-denoising SDEs are constructed using the Riemannian metric, and the exponential map preserves bond-length and angle constraints (Woo et al., 2024).

Conservative and quasi-conservative score fields

Architectural choices affect whether the score field $s_\theta(x)$ is conservative (i.e., can be integrated to a scalar energy). While constrained SBMs (energy-based) guarantee conservativeness, unconstrained SBMs (direct vector field) may introduce curl, impairing sampler efficiency. Quasi-conservative SBMs (QCSBM) append a Hutchinson-trace estimator regularizer to penalize non-conservativeness, achieving a flexible architecture while retaining efficient sampling (Chao et al., 2022).

4. Score-Energy Duality and Hybrid Sampling Schemes

Energy score models are used in diffusion frameworks, which require the drift of the reverse SDE equal to the score of the data distribution. Two main parameterizations exist:

• Score-based diffusion models (SBMs) directly learn $s_\theta(x,t)$ via DSM-style objectives. They enable flexible architectures but lack a closed-form energy function, complicating MCMC accept/reject correction (Aarts et al., 1 Oct 2025).
• Energy-based diffusion models (EBMs) posit a scalar energy $E_\phi(x,t)$, enforcing conservative drifts and enabling direct Metropolis-Hastings acceptance via energy differences, necessary for precise MCMC sampling (Aarts et al., 1 Oct 2025).

Recent work introduces MH-like corrections for score-based models by numerically integrating the score along a proposal path, facilitating MCMC with pre-trained score networks and closing the gap in sampling quality between score- and energy-based samplers (Sjöberg et al., 2023). In deterministic-flow models and TTS, "delta loss" directly matches the score to a first-order displacement vector, optimizing for rapid convergence in few sampling steps (Sun et al., 19 May 2025).

5. Generalizations: Non-Euclidean Noise, Scoring Rules, and Tweedie Extensions

Energy score models generalize classical Tweedie’s formula to broader noise families and leverage proper scoring rules:

• Elliptical and generalized Gaussian noise: The score of the noisy marginal can be expressed as the gradient of an energy score, with the posterior minimizing a strictly proper energy score for each observed $Y$. This is formalized in the Energy–Score identity, which unifies denoising-score and scoring-rule literature and enables robust score estimation under arbitrary noise (Leban, 29 Dec 2025).
• Path-derivative formalism: The gradient of the energy score is implemented by blocking gradients through the posterior model, enabling black-box estimation of Stein scores, robust calibration of noise parameters, and non-Gaussian diffusion samplers (Leban, 29 Dec 2025).

6. Applications and Empirical Performance

Energy score models have demonstrated strong empirical performance across domains:

• High-dimensional image synthesis (CIFAR-10, MNIST): Multi-scale DSM-trained EBMs match GANs in Inception score and FID, with competitive bits/dim and mode coverage, efficient inpainting, and reduced training cost (Li et al., 2019).
• Molecular structure optimization: R-DSM achieves chemical accuracy in energy deviations and RMSD, sharply surpassing classical force-field and DSM baselines. Post-refinement DFT optimizations converge in fewer steps and consume less SCF time (Woo et al., 2024).
• TTS and flow-based models: Score-based and delta-loss training, even in single/few-shot inference, yield perceptually superior synthesis to NCE baselines, with best MOS scores among automated pipelines (Sun et al., 19 May 2025).
• Latent variable EBMs: BiSM and VaES enable tractable training on complex data, maintaining competitive log-likelihoods, Fisher divergences, and sample FID scores (Bao et al., 2020, Bao et al., 2020).
• Sampling efficiency: Hybrid MH-like accept/reject schemes on score-based diffusion models dramatically improve log-likelihood, Wasserstein distance, and variance error, matching or exceeding energy-based MH (Sjöberg et al., 2023).

7. Limitations, Open Problems, and Future Directions

Several open challenges remain for energy score models:

• Non-conservativeness in unconstrained architectures may bias Langevin/SDE sampling; regularization for symmetry, spectral penalties, or adaptive weighting are active research topics (Chao et al., 2022).
• Path integration bias in MH-like corrections (score parameterizations) suggests future work in adaptive path selection and efficient quadrature methods (Sjöberg et al., 2023).
• Calibration and robustness: Generalized energy scores necessitate careful noise parameter estimation and posterior calibration, especially with heavy-tailed or anisotropic noise (Leban, 29 Dec 2025).
• Sampling from intractable posteriors in latent EBMs: While bi-level and variational approaches are consistent under flexible variational families, tuning of inner/outer optimization is non-trivial (Bao et al., 2020).
• Scalability: Extensions to large-scale, multi-modal data, high-dimensional molecular systems, and field-theoretic problems require further exploration of manifold-aware losses, Riemannian MCMC techniques, and efficient surrogate energy estimators (Woo et al., 2024, Aarts et al., 1 Oct 2025).

Energy score models thus form a coherent, theoretically principled, and practically versatile framework for generative modeling, posterior inference, and structure optimization, capable of integrating the strengths of energy-based and score-based paradigms, and advancing state-of-the-art performance across several application domains.