Operator-based KKL (Quantum KL Divergence)

• Operator-based KKL is a divergence measure for density operators and kernel embeddings that extends classical KL divergence using operator convexity.
• It employs variational duality and supremum representations to robustly quantify discrepancies in quantum state distinguishability and nonparametric statistics.
• The framework underpins applications in quantum hypothesis testing, resource theory protocols, and information geometry with efficient quantum algorithmic implementations.

Operator-based Kullback–Leibler (Quantum Kullback–Leibler) divergence generalizes the classical KL measure of discrepancy between probability distributions to settings where objects of interest are operators—particularly density operators in quantum theory and positive definite operators arising from kernel embeddings. Two distinguished and deeply interrelated families arise: the quantum relative entropy for density matrices, including the maximal (Belavkin–Staszewski) variant, and the kernel KL (KKL) divergence on operator embeddings. Both forms exploit operator convexity, variational duality, and provide foundational distances in quantum information, nonparametric statistics, and information geometry.

1. Foundational Definitions and Operator-based Formulations

For quantum states (density operators) $ρ,σ$ on finite-dimensional Hilbert space, the canonical operator-based KL is the quantum relative entropy

$$S(ρ‖σ) = \operatorname{Tr}\bigl[ρ(\logρ - \logσ)\bigr].$$

This reduces to the classical KL divergence when $ρ,σ$ commute, and quantifies the distinguishability of quantum states, operationally characterized by the error exponents in quantum hypothesis testing and rates in resource theory protocols (Felice et al., 2019, Matsumoto, 2013, Lu et al., 13 Jan 2025).

A maximal quantum $f$-divergence is defined as

$$D_f^{\max}(ρ‖σ) = \inf_{\text{reverse tests }(T,p,q)} D_f(p‖q),$$

where the infimum is over all reverse tests—CPTP maps $T$ and classical distributions $p,q$ such that $T(p)=ρ,\,T(q)=σ$ (Matsumoto, 2013). For $f(λ) = λ\logλ$, this recovers the operator-based Kullback–Leibler divergence

$$D_{KL}^{\max}(ρ‖σ) = \operatorname{Tr}\,\rho\,\log(\sigma^{-1/2}\rho\sigma^{-1/2}),$$

also known as the Belavkin–Staszewski entropy $S_{BS}(ρ‖σ)$ (Ortigueira et al., 28 Nov 2025).

For kernel embeddings of distributions $P,Q$ via covariance operators $C_P,C_Q$ in some RKHS $\mathcal{H}$, the kernel KL divergence is

$$D_{KKL}(P\|Q) = \operatorname{tr}[C_P \log C_P - C_P\log C_Q].$$

This is structurally parallel to quantum relative entropy but applied to covariance operators of probability measures (Chazal et al., 2024).

2. Variational and Supremum Representations

Operator KL divergences admit variational dual forms. For $D_f^{\max}$, there is a supremum over pairs of Hermitian operators $(W_1, W_2)$ determined by the operator convex constraint

$r W_1 + W_2 \leq f(r), \quad \forall r \geq 0$

such that

$$D_f^{\max}(ρ‖σ) = \sup_{(W_1,W_2) \in W_{\max}(H)} \{\operatorname{Tr}[ρW_1] + \operatorname{Tr}[σW_2]\}$$

with $f(λ)=λ\logλ$ for the KL case (Matsumoto, 2013). For quantum $f$-divergence estimation on hardware, this variational structure is essential: one reduces $−\log$ to a quadrature over simple $f_t$-divergences, each admitting a variational form whose minima correspond to polynomial operator expectations implementable on NISQ devices (Lu et al., 13 Jan 2025).

3. Key Properties and Comparisons

Operator-based KL divergences possess a suite of crucial properties:

Property	Petz/Umegaki $S(ρ‖σ)$	Maximal $D^{\max}_{KL}(ρ‖σ)$
Data-processing	Yes	Yes
Joint convexity	Yes	Yes
Equality on commuting	Yes	Yes
Additive on tensors	Yes	Yes
Monotonicity	Yes	Yes
Lower semicontinuity	Yes	Yes
Potential negativity	No	Yes (for $σ$ pure)

$D_{KL}^{\max}(ρ‖σ)\ge S(ρ‖σ)$, with equality iff $[ρ,σ]=0$. For $σ=|\psi\rangle\langle\psi|$, $D_{KL}^{\max}(ρ‖σ)$ can be negative and is given by $$\langle\psi|ρ|\psi\rangle\ln\langle\psi|ρ|\psi\rangle$$ (Matsumoto, 2013, Ortigueira et al., 28 Nov 2025).

4. Connections to Classical KL and Ensemble Realizations

Belavkin–Staszewski entropy arises as the minimal KL divergence over all classical ensembles (unravelings) that realize $ρ$ and $σ$. If both are diagonal in a (possibly non-orthogonal) common basis $|\psi_i\rangle$, with $ρ = \sum_i p_i |\psi_i\rangle\langle\psi_i|$, $σ = \sum_i q_i |\psi_i\rangle\langle\psi_i|$,

$$S_{BS}(ρ‖σ) = \sum_i p_i \log(p_i/q_i) = D_{KL}(μ_{CB}‖ν_{CB})$$

where $μ_{CB},ν_{CB}$ are atomic measures on $|\psi_i\rangle$. This identification relates operator-based quantum divergences to classical measure-theoretic KL on the space of pure states and underpins large-deviation theory in quantum ensembles (Ortigueira et al., 28 Nov 2025).

5. Operator-based KL in RKHS: Kernel Kullback–Leibler (KKL) Divergence

The KKL extends operator KL to kernel embeddings: $$D_{KKL}(P\|Q) = \operatorname{tr}[C_P \log C_P - C_P\log C_Q]$$ with $C_P = \int φ(x)\otimes φ(x)\,dP(x)$ in RKHS. $D_{KKL}$ interpolates between classical KL and smoothed KL; it can be lower-bounded by kernel-smoothed KL. Notably, the unregularized KKL may not be defined if supports are disjoint; introducing regularization or “skew” variants ensures well-definedness: $$D_{KKL,λ}(P\|Q) = \operatorname{tr}[C_P(C_Q+λI)^{-1}] - \log\det[C_P(C_Q+λI)^{-1}] - (\text{n}_P - \text{n}_Q)$$ which is always finite for full-rank $C_Q+λI$ (Chazal et al., 2024).

6. Algorithmic Estimation on Quantum Hardware

Quantum algorithms for operator-based KL estimation (as in (Lu et al., 13 Jan 2025)) proceed by

• decomposing $-\log$ via high-accuracy quadrature into $f_t$ divergences,
• representing variational minima via parameterized Hermitian polynomials,
• estimating trace functionals on quantum circuits using “extended SWAP-test” schemes,
• assembling the final result through classical optimization.

This approach enables estimation using at most $2n+1$ qubits (for $n$-qubit inputs), distributed evaluation across hardware, and yields efficient $O(\text{poly}(n))$ scaling. Error rates can be directly controlled by quadrature nodes and optimization precision.

7. Geometric and Information-theoretic Interpretation

Quantum KL functions as the canonical divergence in the information geometry of density operators. On the manifold of quantum states endowed with the quantum Fisher metric, the operator-based divergence is the matrix Bregman divergence associated to free energy. In the kernel/RKHS setting, KKL inherits strict convexity and Bregman structure, enabling Wasserstein gradient flows with properties analogous to standard KL-based flows but with superior support-mismatch sensitivity compared to first-moment metrics like MMD. As a measure of complexity and many-body correlation, quantum KL encapsulates the divergence from exponential (Gibbs) families, aligning with projections in exponential families and providing a fundamental information-geometric measure for both quantum and classical systems (Felice et al., 2019, Chazal et al., 2024).

---

The theory of operator-based Kullback–Leibler divergence establishes a unified perspective on quantum information metrics across quantum physics and nonparametric statistics, centering on the variational, geometric, analytic, and algorithmic properties of operator KL measures. Its maximal and regularized variants grant operational flexibility and foundational robustness in quantum information tasks and modern machine learning with operator-valued data.