---
title: Unified Formula for Relative Entropy
url: https://www.emergentmind.com/topics/unified-formula-for-relative-entropy
type: topic
---

# Unified Formula for Relative Entropy

The unified formula for relative entropy provides a mathematical framework that simultaneously generalizes standard divergence measures, interpolates between multiple entropy-like quantities, and allows for a systematic analysis of quantum, classical, and resource-theoretic distinguishability. The construction encapsulates diverse contexts—quantum information, statistical mechanics, kinetic theory, field theory, and operational resource theories—using core variational, integral, or operator-functional expressions that reduce to Umegaki (von Neumann), Rényi, and Tsallis relative entropies and their physical or operational generalizations.

## 1. Canonical Definition and Variational Frameworks

The most fundamental quantum relative entropy between density operators $\rho, \sigma$ on a Hilbert space $\mathcal{H}$ is the Umegaki entropy:
\[
D(\rho\Vert\sigma) := \operatorname{Tr}\left[\rho(\log\rho - \log\sigma)\right],
\]
defined when $\operatorname{supp}\rho\subseteq\operatorname{supp}\sigma$, and $+\infty$ otherwise. This form unifies the classical Kullback–Leibler divergence and subsumes prominent entropic quantities used across information theory and statistical physics [2208.12194][1309.4046].

Unified functional characterizations extend this definition in two principal directions:
- **Parametric (α,β) Families**: A two-parameter unified quantum relative entropy for $\alpha\in[0,1]$, $\beta\in\mathbb{R}$,
  \[
  D_{\alpha}^{\beta}(\rho\Vert\sigma) =
  \frac{1}{(\alpha-1)\beta} \left[\, \left(\operatorname{Tr}\rho^{\alpha}\sigma^{1-\alpha}\right)^{\beta} - 1\,\right], \quad \beta\neq0,\,\alpha<1,
  \]
  with appropriate limiting cases for $\beta\to0$ (Rényi), $\beta=1$ (Tsallis), and $\alpha\to1$ (Umegaki) [2506.09779][2506.09799]. These families interpolate smoothly between additive/logarithmic and non-additive/power-law divergences, ensuring broad applicability.

- **Convex-Functional and Variational Principles**: For convex $\varphi$, $D_\varphi(\rho\Vert\sigma) := \operatorname{Tr}[\varphi(\rho) - \varphi(\sigma) - \varphi'(\sigma)(\rho-\sigma)]$, with operator-monotonicity of $\varphi'$ ensuring monotonicity under completely positive trace-preserving maps [1309.4046].

Additionally, variational frameworks define a divergence $\Psi_s(\omega\Vert\sigma)$ for normal states $\omega,\sigma$ on a von Neumann algebra, such that
\[
\Psi_{s}(\omega\Vert\sigma) = \inf_{x} \left\{\, C_{p} \int_{0}^\infty \left[\omega(x(t)^*x(t)) + t^{-1}F((1-x(t))\sigma(1-x(t))^*)\right] t^{-2/p}\,dt \right\} - 1,
\]
recovering, in the limits $s\to1$, $s\to\frac12$, and $s\in(\tfrac12,1)$, the Umegaki entropy, fidelity, and sandwiched Rényi entropy, respectively [2009.05024].

## 2. Unified Integral and Distribution-Based Representations

Integral and spectral representations provide a unifying route by expressing relative entropies in terms of observable distributions. For quantum states $\rho$, $\sigma$:
\[
D(\rho\Vert\sigma) = \int_{-\infty}^{+\infty} \frac{|t|}{(t-1)^2} \operatorname{Tr}_-[(1-t)\rho + t\sigma]\,dt,
\]
where $\operatorname{Tr}_-$ is the sum over negative eigenvalues—yielding the classical Kullback–Leibler divergence when $\rho$, $\sigma$ commute [2208.12194].

For Rényi entropy, a distribution-of-observables construct applies:
\[
D_\alpha(\rho\Vert\sigma) = \frac{1}{\alpha-1}\log\left\langle \sigma^{1-\alpha}\right\rangle_{\rho^\alpha},
\]
with expectations defined via the pushforward spectral measure. The Umegaki entropy fits as:
\[
D(\rho\Vert\sigma) = \left\langle -\log\sigma \right\rangle_\rho - \left\langle -\log\rho \right\rangle_\rho,
\]
applicable in both finite and infinite dimensions, including Fock spaces, and for continuous variables [2203.01964].

## 3. General Properties and Special Cases

The unified formulas yield the following universal properties for $D(\cdot\Vert\cdot)$ and its generalizations [1309.4046][2506.09799]:
- **Nonnegativity and Faithfulness**: $D(\rho\Vert\sigma)\geq0$, with equality iff $\rho=\sigma$.
- **Data Processing Inequality (DPI)**: $D(\Lambda(\rho)\Vert\Lambda(\sigma)) \leq D(\rho\Vert\sigma)$ for any quantum channel $\Lambda$ (CPTP map), for the Umegaki, Rényi ($\beta\leq1$), and Tsallis (for $0<\alpha<1$) entropies.
- **Continuity and Limits**: The families are continuous in both $\alpha,\beta$, and $\rho,\sigma$; correct limiting behavior interpolates between classical and quantum divergences [2506.09779][2506.09799].
- **Convexity and Additivity**: Joint convexity in the state arguments; additivity for the logarithmic (Rényi, Umegaki) cases, with pseudo-additivity in the Tsallis regime.
- **Infinite Dimensions**: Provided the operator monotonicity condition holds, monotonicity and semicontinuity extend to infinite dimensions [1309.4046][2203.01964].
- **Functional Inequalities**: Stochastic control formulations unify Talagrand’s transport, log-Sobolev, Poincaré, and other classical inequalities as minimization problems for a drift generating the target distribution [1006.3028].

## 4. Physical and Mathematical Applications

Unified entropy formulas underpin a broad class of applications:
- **Quantum Statistical Mechanics**: Irreversibility, measurement of quantum distinguishability, resource-theoretic quantifiers.
- **Continuous Variable & Gaussian Systems**: Gaussian relative entropy is decomposed into classical (Shannon-like) and quantum (covariance/mean) terms, with explicit closed forms for $n$-mode states [2102.06708].
- **Quantum Field Theory (QFT)**: The modular Hamiltonian structure, symplectic-flux formulae, and bulk-boundary correspondences yield holographic and conformal field theoretical expressions for relative entropy [1512.06431][1904.08433].
- **Macroscopic Kinetic Theory**: In kinetic equations (Vlasov–Fokker–Planck), a modulated relative entropy functional combines microscopic Maxwell–Boltzmann proximity and macroscopic density deviation via nonlocal (Riesz) interaction energies, ensuring control over convergence in both strong and weak topologies [2510.17455].
- **Resource Theories**: Unified $(\alpha,\beta)$–relative entropies form the operational basis for measures in coherence, imaginarity, and related resource monotones [2506.09799].

## 5. Specialized Structures: Entanglement, Modularity, Boundary Phenomena

The unified formula applies to more structured cases:
- **Entanglement**: The relative entropy of entanglement is given by minimizing $S(\rho\Vert\sigma)$ over all separable states, with a universal closed-form constructed from the structure of the closest separable state (CSS) and supporting hyperplanes [1007.4544].
- **Modular Theory (Algebraic QFT, CCR Algebras)**: On CCR algebras, the relative entropy between quasifree/coherent states is a quadratic form of the difference in displacement vectors, computed using modular data and projectors associated with standard subspaces [2012.14401].
- **Boundary and Left-Right Decompositions in CFT**: The left-right relative entropy of boundary states in CFTs is computed via explicit sums over modular S-matrix data, conformal weights, and Ishibashi coefficients, with cutoff-independent universal formulas. The notion of “relative entanglement sectors” organizes states indistinguishable by this entropy according to symmetry/NIM-rep structure [2411.09406].

## 6. Comparison and Synthesis of Unification Principles

All unified formulas share a common functional and structural DNA:
- They interpolate analytically between core entropic quantities (Umegaki, Rényi, Tsallis, and operational monotones).
- Spectral and integral representations turn operator divergences into trace-based, “classicalized” expressions, extending applicability to infinite dimensions and general probabilistic theories (GPTs) [2208.12194][2203.01964].
- Variational, convex-analytic, and stochastic control frameworks convert operational or resource-theoretic divergence minimization problems into integral or supremum representations [2009.05024][1006.3028].
- The same operator and variational calculus controls quantum coherence/imaginarity measures, resource monotones, and field-theoretic modular flows [2506.09799][1512.06431].

The core unification thus lies in expressing quantum, classical, and operational distinguishability through a minimal set of analytic, variational, or distribution-theoretic objects that manifestly encode physical, probabilistic, or resource-theoretic monotonicity and convexity principles.

Source: https://www.emergentmind.com/topics/unified-formula-for-relative-entropy