Variational Entropic Formulation

• Variational entropic formulation is a mathematical framework that optimizes states by balancing energy, cost, and entropy.
• It unifies principles across statistical mechanics, stochastic processes, and optimal transport through variational methods.
• The framework enables rigorous analysis and efficient algorithms in fields such as quantum mechanics, density functional theory, and machine learning.

A variational entropic formulation refers to a class of principles and mathematical frameworks in which target states, paths, or distributions are identified as optimizers (minimizers or maximizers) of a functional involving an entropy term (or relative entropy/divergence) alongside energy, constraint, or cost terms. Such formulations have deep connections and unifying roles across statistical mechanics (both classical and quantum), stochastic processes, optimal transport, variational inference, thermodynamics, PDE theory, and free probability.

1. Core Concept: Variational Principles with Entropic Functionals

The archetypal variational entropic formulation is the Gibbs (or Donsker–Varadhan) principle, expressing functionals of the form $-\log \mathbb{E}_\nu[e^{-f}]$ as minimizations over probability measures: $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$ where $H(\mu \|\nu)$ denotes relative entropy. This generalizes to a wide spectrum of settings, including infinite-dimensional Wiener spaces, diffusions, quantum systems (using quantum relative entropy), and beyond (Hartmann, 2016, Hartmann, 2016, 2002.04253).

Such functionals arise naturally whenever exponential integrability, large deviations, or equilibrium statistical mechanics play a role. They characterize stationary points, evolution equations, or optimal controls as minimizers/maximizers of appropriately constructed entropic functionals.

2. Quantum and Classical Statistical Mechanics: Gibbs Variational Characterization

In equilibrium quantum statistical mechanics, the variational entropic formulation appears as the Gibbs variational formula: $$p(\beta) = \sup_{\omega \in \mathcal{S}^\tau} \left\{ -\beta\,e(\omega) - S(\omega \| \omega_0) \right\},$$ where $p(\beta)$ is the pressure, $e(\omega)$ the energy density, and $S(\omega \| \omega_0)$ the quantum relative entropy density with respect to a reference state. The supremum runs over translation-invariant states (2002.04253).

This extends Föllmer’s entropy principle for classical lattice systems, fully characterizing equilibrium/KMS states as entropy–energy optimizers within the translation-invariant state space. Notably, uniqueness of the equilibrium state is not required—the formulation applies in the presence of phase coexistence and multiple extremal states.

Key features:

• The pressure is identified with an entropy–energy supremum.
• The functional $-\beta e(\omega) - S(\omega \| \omega_0)$ quantifies the tradeoff between energy and informational cost.
• All equilibrium (DLR/KMS) states are precisely the maximizers, unifying the conceptual landscape.

3. Stochastic Processes and Wiener Space: Entropic Variational Formulas

In stochastic analysis, variational entropic representations generalize to functionals of Brownian motion, diffusions, and Wiener space measures. The Boué–Dupuis formula and its extensions state: $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{u \in \mathcal{D}} \left\{ \frac{1}{2}\mathbb{E}_\nu \|u\|_H^2 + \mathbb{E}_\nu[f(W^u)] \right\}$$ where $W^u$ is an adapted Cameron–Martin shift of the Brownian path, and the minimization captures both entropy (through Girsanov’s formula) and cost in $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$0 (Hartmann, 2016, Hartmann, 2016, Hartmann, 2016).

For conditional expectations, the extended form is

$$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$1

This robustly applies to Brownian bridge measures, small-noise diffusions, loop measures, and systems like Dyson Brownian motion (Hartmann, 2016).

The entropic criterion for invertibility of shifts is that exact equality holds between entropy and quadratic energy: $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$2 implying the existence of a strong solution to the associated SDE and linking the variational problem to strong existence by entropy methods.

4. Variational Entropic Formulation in Density Functional Theory (DFT)

DFT, central in quantum chemistry and condensed matter, is fully recoverable as an entropic variational principle. Both in classical and quantum settings, trial distributions (or density matrices) constrained to match the one-body density profile $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$3 maximize the (quantum) relative entropy subject to energy and density constraints. The stationarity of the resulting entropic functional yields the Euler–Lagrange equation that constitutes the DFT variational principle (Yousefi et al., 2022, Yousefi, 2021).

Concretely, for the quantum case: $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$4 with Lagrange multipliers $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$5 enforcing the density constraint, and the corresponding free energy functional $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$6 arising from a Legendre transform of the entropy.

Approximation schemes (mean-field, LDA, Kohn–Sham) are interpreted as modeling choices for the intrinsic free-energy part of $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$7 in the entropic functional framework.

5. Entropic Variational Formulations in Optimal Transport

Entropic regularization of optimal transport is naturally formulated via variational entropic functionals. For probability measures $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$8, the entropic OT problem seeks a coupling $$-\log \mathbb{E}_\nu[e^{-f}] = \inf_{\mu \ll \nu} \left\{ \mathbb{E}_\mu[f] + H(\mu \| \nu) \right\},$$9 minimizing

$H(\mu \|\nu)$0

and admits a dual formulation involving the log-partition function, itself recast via a variational upper bound (Dyachenko et al., 2 Feb 2026). Sampling-free optimization becomes tractable by introducing an auxiliary variable, giving

$H(\mu \|\nu)$1

All finite-sample and approximation error analyses occur within this variational–entropic framework.

In dynamic settings, the entropic-regularized dynamic OT problem is encoded as a constraint Hamiltonian system (McKean–Pontryagin formulation), with Lagrange multipliers enforcing entropic diffusion regularization (Reich, 31 Mar 2026).

The same variational principle enables foundational connections to large deviations, Gamma-convergence, and gradient flows in the Wasserstein space (Peletier et al., 2011, Karatzas et al., 2018).

6. Variational Entropic Formulations in Non-Equilibrium and Stochastic Thermodynamics

In finite-dimensional stochastic thermodynamics, the fully general variational formulation addresses coupled mechanical and thermal variables. An augmented Lagrange–d'Alembert principle yields coupled SDE/ODE systems for mechanical and entropy variables, with entropy production entering directly as a variational constraint (Pino et al., 2 Oct 2025, Carlier, 2024).

Irreversible dynamics are encoded via nonholonomic constraints and metriplectic brackets, enforcing the second law (nonnegative entropy production) and forcing fluctuation–dissipation relations and Onsager symmetry. The entropy functional acts as a generator of irreversible flow in the unified mechanical + thermodynamic phase space.

7. Advanced Topics and Generalizations

• Free probability: Operations such as free convolution have entropic variational characterizations, where kernel integrals become supremums of log-potentials minus relative entropy, leading to new inequalities and explicit solutions (Arizmendi et al., 2023).
• Quantum information: Variational entropic expressions for relative entropy and derived divergences are fundamental to quantum information theory and operator algebras, characterizing monotonicity, joint convexity, and certainty relations (Hollands, 2020).
• Variational entropy in conservation laws: Formulations for scalar conservation laws generalize total-variation principles using seminorms of the gradient as "variation entropies," providing new stability concepts closely tied to entropy conditions (Eikelder et al., 2018).
• Variational inference and learning: ELBO and related bounds at stationarity reduce to explicit entropy sums, exposing the underlying variational–entropic structure in black-box variational learning (Lücke et al., 2022). New directions consider maxitive possibility-theoretic analogs of classic variational bounds (Singh et al., 26 Nov 2025).

Table: Representative Forms of Variational Entropic Principles

Domain	Key Variational-Entropic Principle	Main Reference
Quantum stats. mech.	$H(\mu \|\nu)$2	(2002.04253)
Wiener space/SPDEs	$H(\mu \|\nu)$3	(Hartmann, 2016, Hartmann, 2016)
Density Functional	$H(\mu \|\nu)$4	(Yousefi et al., 2022, Yousefi, 2021)
Entropic OT	$H(\mu \|\nu)$5 (dual: maximization involving partition function)	(Dyachenko et al., 2 Feb 2026)
Stochastic thermodyn.	SDE/ODE system from entropy-augmented variational principle	(Pino et al., 2 Oct 2025, Carlier, 2024)
Free probability	$H(\mu \|\nu)$6	(Arizmendi et al., 2023)

8. Summary and Unifying Perspective

Variational entropic formulations provide a unifying architecture for an array of theories in statistical physics, probability, PDE theory, stochastic analysis, and learning. In each context, they encode the optimal compromise between a driving "energy" (including cost, internal energy, or likelihood) and an entropic penalty quantifying deviation from a baseline or prior. Their technical utility lies in certifying optimality, providing existence/uniqueness theorems, encoding physical laws (such as the second law or maximum entropy), and generating algorithmic procedures for sampling, learning, and inference.

These constructions not only clarify the mathematical structure underlying key theories but also decouple the core principles from any particular implementational framework, revealing deep commonalities across otherwise disparate domains (2002.04253, Hartmann, 2016, Hartmann, 2016, Carlier, 2024, Pino et al., 2 Oct 2025, Yousefi et al., 2022, Yousefi, 2021, Arizmendi et al., 2023, Dyachenko et al., 2 Feb 2026).