Computational Relative Entropy

• Computational relative entropy is a family of methods extending traditional KL divergence by incorporating resource constraints and algorithmic limitations.
• It employs numerical estimators, semidefinite programming, and variational techniques to analyze statistical, quantum, and field-theoretic data.
• By linking operational distinguishability with computational feasibility, it offers practical insights for hypothesis testing, channel capacity, and cryptographic protocols.

Computational relative entropy is not a single formalism but a family of computational viewpoints on relative entropy and related divergences. Across statistics, quantum information, field theory, holography, cryptography, and compression, it denotes either the explicit computation of relative entropy from data, operators, or path integrals, or a complexity-aware replacement of ordinary relative entropy in which distinguishability is restricted by computational resources. In its standard form, relative entropy is the Kullback–Leibler divergence or its quantum analogue,

$$D_{\mathrm{KL}}(P\|Q)=\sum_i p_i\log\frac{p_i}{q_i},\qquad S(\rho\|\sigma)=\operatorname{Tr}[\rho(\log\rho-\log\sigma)],$$

while computational variants modify either the admissible tests, the optimization domain, or the numerical procedure used to evaluate these quantities (0808.4111, Koßmann et al., 2024, Meyer et al., 24 Sep 2025).

1. Definitions, scope, and basic distinctions

The classical discrete form used throughout statistics is

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$

with finiteness iff $P$ is absolutely continuous with respect to $Q$. In finite-dimensional quantum theory the corresponding quantity is

$$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$

and the max-relative entropy is

$$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$

For quantum channels, the stabilized channel relative entropy is

$$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$

with $\Gamma^\mathcal{N}_{AB}$ the Choi matrix (Koßmann et al., 2024).

A separate usage appears in complexity theory and cryptography. There, computational relative entropy does not merely mean “relative entropy computed numerically”; it is an operational quantity defined under bounded resources. One such formulation is the optimal type-II error exponent in asymmetric hypothesis testing when only polynomially many copies and polynomially many quantum gates are allowed:

$$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$

where $D_h^\epsilon$ is the complexity-limited hypothesis-testing relative entropy (Meyer et al., 24 Sep 2025).

A third usage is methodological: the numerical estimation of KL or quantum relative entropy from samples, covariance operators, semidefinite relaxations, or path integrals. This includes trace-log formulas for Gaussian fields, MCMC-based evidence estimators, semidefinite approximations of matrix logarithms, quadrature-based quantum algorithms, and Euclidean replica constructions in CFT (Schröfl et al., 2023, Mehrabi et al., 2019, Fawzi et al., 2017, Lu et al., 13 Jan 2025, Lashkari, 2014).

This suggests that “computational relative entropy” has at least three stable meanings: resource-bounded distinguishability, computable surrogate formulations, and practical numerical algorithms. A plausible implication is that the term now functions as a unifying interface between operational information measures and the algorithmic constraints of specific domains.

2. Complexity-constrained relative entropy

In the complexity-theoretic formulation, the central object is not $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$0 itself but the best asymptotic discrimination rate achievable by restricted tests. Measurements of gate complexity at most $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$1 are modeled by effects in $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$2, and the corresponding single-shot quantity is

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$3

The resulting regularized quantity satisfies, up to negligible terms,

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$4

is additive under polynomial tensor powers, obeys data processing for poly-complexity channels, and has a computational Stein’s lemma identifying it with a computationally measured two-outcome relative entropy under a polynomial bound on measured max-relative entropy (Meyer et al., 24 Sep 2025).

The same framework yields computational analogues of Pinsker and Bretagnolle–Huber inequalities:

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$5

and a computational smoothing property,

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$6

so computationally indistinguishable states have equivalent complexity-aware information measures (Meyer et al., 24 Sep 2025).

Earlier cryptographic work developed related KL-based hardness notions for search problems. Hardness in relative entropy is defined through the KL divergence between the joint law of coins and outputs produced by an online generator and a simulated alternative:

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$7

If a distributional search problem is $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$8-hard, then this divergence is at least $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$9, and the same chain-rule structure yields next-block pseudoentropy and next-block inaccessible entropy (Agrawal et al., 2019).

Convex analysis gives a complementary perspective. A restricted or Editor's term “test-limited” relative entropy arises by restricting the variational characterization of KL to a class of efficient functions. In the paper’s convex-dual viewpoint,

$P$0

and computational entropy becomes a separation problem between a target distribution and a convex entropy superlevel set, tested only by efficient distinguishers. For metric min-entropy, deterministic boolean and deterministic real-valued distinguishers coincide, whereas for finite Rényi order randomized or real-valued distinguishers are strictly stronger than deterministic boolean ones (Skórski, 2013).

These results establish that computational relative entropy is not merely an approximation to ordinary relative entropy. It is a distinct operational invariant whose value can collapse to zero even when the unbounded relative entropy is infinite, as in pseudorandom or one-way-function-based constructions (Meyer et al., 24 Sep 2025, Agrawal et al., 2019).

3. Convex, variational, and semidefinite formulations

A major line of work treats relative entropy as an optimization primitive. In finite-dimensional quantum information, many quantities reduce to minimization or maximization of $P$1 over convex sets. Examples include the relative entropy of entanglement,

$P$2

its PPT relaxation,

$P$3

the Holevo information written as

$P$4

and channel optimization problems expressed via conditional entropy or relative entropy of recovery (Fawzi et al., 2017).

The numerical obstacle is the matrix logarithm. A unified SDP approach replaces $P$5 by semidefinite approximations built from the integral representation

$P$6

followed by high-order quadrature and scaling-and-squaring. In the implementation described in CvxQuad, quantum_entr, trace_logm, quantum_rel_entr, quantum_cond_entr, and op_rel_entr expose these approximations to off-the-shelf semidefinite solvers. The paper reports absolute errors around $P$7 for simple cq channels with default $P$8 and uses the framework to compute relative entropy of entanglement, entanglement-assisted capacity, degradable-channel quantum capacity, and relative entropy of recovery (Fawzi et al., 2017).

For channel relative entropy, a distinct semidefinite framework starts from the Frenkel–Jencová operator-integral representation

$P$9

Substituting $Q$0 and

$Q$1

produces lower- and upper-bound SDPs that sandwich $Q$2 with any desired precision. The lower-bound SDP optimizes over $Q$3 and auxiliary variables $Q$4, while the upper-bound SDP uses variables $Q$5, $Q$6, and $Q$7. The number of grid points needed for an $Q$8-approximation scales as

$Q$9

with $$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$0 (Koßmann et al., 2024).

A related variational strategy appears for state estimation on quantum hardware. There, $$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$1 and Petz Rényi divergences are reduced to weighted sums of $$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$2 via quadrature formulas for $$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$3 and $$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$4, and each $$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$5 is estimated through a variational minimization over an operator $$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$6 parameterized by shallow circuits. The method uses only copies of $$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$7 and $$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$8, an ancilla, controlled unitaries, and an extended SWAP test, with total circuit width at most $$D(\rho\Vert\sigma)= \begin{cases} \operatorname{Tr}[\rho(\ln\rho-\ln\sigma)], & \text{if } \operatorname{supp}(\rho)\subseteq \operatorname{supp}(\sigma),\ +\infty,& \text{otherwise}, \end{cases}$$9 qubits (Lu et al., 13 Jan 2025).

Across these formulations, a common pattern emerges: relative entropy is made computationally tractable by replacing nonlinear logarithms with integral, rational, or variational surrogates that preserve convexity or monotonicity sufficiently well for optimization.

4. Statistical estimation, Gaussian theories, and sample-based computation

In classical statistics, computational relative entropy begins with direct evaluation from counts. For i.i.d. discrete data with empirical frequencies $$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$0,

$$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$1

with complexity $$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$2. The identity

$$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$3

makes KL the central functional of hypothesis testing, maximum likelihood, model selection, maximum entropy reconstruction, the EM algorithm, and Markov order determination (0808.4111).

For Gaussian scalar field theories, the computation is spectral. If $$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$4 and the measures are equivalent, then

$$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$5

and in an eigenbasis of $$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$6 with eigenvalues $$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$7,

$$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$8

When only the mass changes, with common Laplacian eigenvalues $$D_{\max}(\rho\Vert\sigma)\coloneqq \ln \inf\{\gamma\ge 0:\ \rho\le \gamma\,\sigma\}.$$9,

$$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$0

The paper proves that for different masses with the same classical boundary conditions, $$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$1 is finite in finite volume iff $$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$2, while Dirichlet versus Robin boundary conditions are mutually singular for all $$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$3 (Schröfl et al., 2023).

The same paper treats mutual information between disjoint regions as a KL divergence between the joint Gaussian field and the product of restrictions:

$$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$4

If $$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$5, then the mutual information is finite; it satisfies an area law,

$$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$6

and becomes infinite for touching regions. In one dimension,

$$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$7

This is a fully explicit field-theoretic computation, but it is also a numerical recipe: discretize the Laplacian, build covariance blocks, and compute log-determinants (Schröfl et al., 2023).

Sample-based Bayesian estimation gives another computational regime. For posterior-to-prior information gain,

$$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$8

so one needs posterior samples, likelihood evaluations, and an evidence estimate. The paper estimates the evidence via a k-nearest-neighbor density-ratio method:

$$D(\mathcal{N}\Vert \mathcal{M}) = \sup_{\rho_A\in\mathcal{S}(\mathcal{H}_A)} D\!\Big(\rho_A^{1/2}\,\Gamma^\mathcal{N}_{AB}\,\rho_A^{1/2}\,\Big\Vert\,\rho_A^{1/2}\,\Gamma^\mathcal{M}_{AB}\,\rho_A^{1/2}\Big),$$9

then forms

$\Gamma^\mathcal{N}_{AB}$0

In the linear Gaussian model, the reported relative error is below $\Gamma^\mathcal{N}_{AB}$1 for sample size larger than $\Gamma^\mathcal{N}_{AB}$2 (Mehrabi et al., 2019).

Random-state calculations occupy an intermediate position between exact formulas and asymptotic estimation. For reduced states from the Wishart ensemble with aspect ratio $\Gamma^\mathcal{N}_{AB}$3, the large-$\Gamma^\mathcal{N}_{AB}$4 average relative entropy is

$\Gamma^\mathcal{N}_{AB}$5

derived from replica limits of hypergeometric expressions for $\Gamma^\mathcal{N}_{AB}$6 and $\Gamma^\mathcal{N}_{AB}$7. This provides a closed-form estimator for chaotic many-body eigenstates and black-hole microstate distinguishability (Kudler-Flam, 2021).

5. Quantum many-body, field-theoretic, and holographic computations

In conformal and algebraic quantum field theory, computational relative entropy often means an explicit evaluation of Araki or Umegaki relative entropy through modular theory, replica limits, or quasi-free formulas.

For 1+1-dimensional CFT, the Euclidean path-integral construction computes sandwiched Rényi relative entropies via replicated manifolds. For a holomorphic vertex operator $\Gamma^\mathcal{N}_{AB}$8 on a circle, the replicated correlator yields

$\Gamma^\mathcal{N}_{AB}$9

and analytic continuation gives

$$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$0

For thermal states at temperatures $$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$1 and $$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$2 on an interval of length $$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$3, with $$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$4,

$$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$5

The relative entropy is UV-finite because the short-distance divergences cancel in the replica ratios (Lashkari, 2014).

For chiral free fermions, Araki relative entropy reduces to a one-particle trace formula for quasi-free CAR states. The mutual information of two disjoint intervals can be written as

$$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$6

with $$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$7 built from $$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$8 and its compressions to the one-particle subspaces. The exact answer for $$\underline{D}(\rho_n \Vert \sigma_n) :\!\simeq \lim_{\epsilon\to0}\lim_{\ell\to\infty}\liminf_{k\to\infty} \frac{1}{n^k} D_h^\epsilon(\rho_n^{\otimes n^k}\Vert \sigma_n^{\otimes n^k};n^{k\ell}),$$9 free chiral fermions is

$D_h^\epsilon$0

and for two intervals this becomes the standard cross-ratio logarithm. For finite-index subnets, the short-distance asymptotics picks up an additive constant $D_h^\epsilon$1 controlled by the global dimension (Longo et al., 2017).

For fermionic QFT on globally hyperbolic spacetimes, the self-dual CAR framework yields a genuinely modular-theoretic formula. If $D_h^\epsilon$2 is a faithful quasifree state, $D_h^\epsilon$3 is the one-particle modular flow, and $D_h^\epsilon$4 is a unitary field excitation with $D_h^\epsilon$5 and $D_h^\epsilon$6, then

$D_h^\epsilon$7

For a $D_h^\epsilon$8-KMS state on an ultrastatic spacetime,

$D_h^\epsilon$9

and in a mode basis

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$00

This is the fermionic analogue of coherent-state relative entropy in free scalar QFT (Galanda, 2022).

In holography, relative entropy becomes geometrizable. For a boundary region $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$01,

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$02

and the central semiclassical statement is

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$03

The modular Hamiltonian maps to bulk operators as

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$04

For ball-shaped regions in the vacuum,

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$05

and the bulk relative entropy reduces, in local modular-Hamiltonian cases, to canonical energy. The 2013 holographic analysis established $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$06 at first order and positivity at second order for broad classes of perturbations, while the 2015 analysis promoted this to equality between boundary and bulk relative entropy inside the entanglement wedge (Blanco et al., 2013, Jafferis et al., 2015).

6. Communication, compression, and algorithmic applications

In relative entropy coding, computational relative entropy becomes a design principle for exact one-shot compression with shared randomness. A channel simulation algorithm communicates a sample $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$07 using a code whose expected length is lower-bounded by mutual information:

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$08

An exact relative entropy coding algorithm attains

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$09

To sharpen the lower bound, the thesis introduces the Channel Simulation Divergence $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$10 and proves

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$11

The computational complexity of selection samplers is controlled by $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$12:

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$13

A* sampling and Greedy Poisson Rejection Sampling attain code lengths within the optimal $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$14 overhead (Flamich, 19 Jun 2025).

The same work gives a concrete continuous-space recipe. For Gaussian channels, if $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$15 and $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$16, then

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$17

and A* or GPRS yields

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$18

This is presented as a quantization-free compression method that integrates naturally with reparameterizable probabilistic models (Flamich, 19 Jun 2025).

Bayesian implicit neural representations instantiate the same principle in learned codecs. With posterior $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$19 and coding distribution $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$20, the training objective is

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$21

The empirical claim in the thesis is that REC with Bayesian INRs attains competitive or superior rate–distortion performance while using small, energy-efficient models (Flamich, 19 Jun 2025).

At a more abstract level, complete diagrammatic axiomatisations have recently been given for KL and Rényi divergences on categories of finite stochastic matrices. In this setting, KL on distributions is

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$22

and its column-wise extension to stochastic matrices is

$$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$23

The computational content comes from exact chain rules and max laws encoded as quantitative equations on string diagrams, yielding compositional calculation rules for probabilistic circuits (Sarkis et al., 4 Mar 2026).

This suggests that algorithmic applications of relative entropy now extend well beyond inference and discrimination. They include source coding on continuous spaces, compositional reasoning for stochastic kernels, and operational rates for privacy- or perception-constrained generative mechanisms.

7. Structural themes, misconceptions, and limitations

A common misconception is that computational relative entropy is synonymous with approximate KL evaluation. The literature instead supports two distinct claims. First, there are practical algorithms for estimating ordinary relative entropy from samples, matrices, channels, or path integrals. Second, there are genuinely new complexity-aware divergences whose operational content differs sharply from the unbounded theory (Mehrabi et al., 2019, Fawzi et al., 2017, Meyer et al., 24 Sep 2025).

Another misconception is that computational restrictions merely perturb relative entropy by small errors. Complexity-theoretic separations show otherwise. Under post-quantum one-way functions, there exist efficiently preparable classical states with computational relative entropy asymptotically zero while the ordinary relative entropy is infinite. Under classically-hard but quantumly-easy one-way functions, one obtains a quantum–classical gap: quantum observers can distinguish states with positive computational exponent whereas classical observers cannot (Meyer et al., 24 Sep 2025).

In numerical work, limitations are domain-specific. Gaussian-field formulas require equivalence of measures or regularized Fredholm determinants; in $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$24 the different-mass KL divergence diverges. MCMC estimators suffer from curse-of-dimensionality effects in kNN density estimation and from poor overlap between posteriors. SDP methods scale poorly with Hilbert-space dimension because matrix-log approximations introduce lifted blocks of size on the order of $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$25. Channel-relative-entropy SDPs depend on $$D_{\mathrm{KL}}(P\,\|\,Q)=\sum_{i=1}^m p_i \log\frac{p_i}{q_i},$$26, so ill-conditioned channel pairs require finer grids. Variational quantum estimators require expressive ansätze and sufficient shot budgets. Replica constructions in CFT are highly explicit in 1+1 dimensions but much harder in higher dimensions or on higher-genus replica manifolds (Schröfl et al., 2023, Mehrabi et al., 2019, Fawzi et al., 2017, Koßmann et al., 2024, Lu et al., 13 Jan 2025, Lashkari, 2014).

A further subtlety concerns monotonicity and cancellation. In gauge theories and gravity, local edge-mode or surface terms can change entanglement entropy, but relative entropy is often insensitive because those local contributions cancel between the two states being compared. This cancellation is central both in bulk–boundary relative entropy and in gauge-theoretic subregion constructions (Jafferis et al., 2015).

Taken together, these works define computational relative entropy as a broad research program rather than a single formula. Its unifying principle is operational: distinguishability is quantified not only by states and hypotheses, but also by what can be computed, optimized, or encoded under concrete structural constraints.