Determinant-Based Mutual Information

• Determinant-based mutual information is a framework that represents dependence using log-determinant and determinant ratio constructions derived from second-order statistics.
• This approach yields exact mutual information expressions for Gaussian models and provides sharp lower bounds in non-Gaussian cases through regression-error determinants.
• Applications span blind source separation, multi-view learning, and spatial field analysis, enabling robust techniques in signal processing and machine learning.

Searching arXiv for the cited papers and closely related determinant-based mutual information work. arxiv_search query: "(Erdogan, 2022) determinant mutual information log determinant entropy blind source separation" arxiv_search results requested for:

• (Erdogan, 2022)
• (Bowsher et al., 2014)
• (Skean et al., 2023)
• (Wan et al., 2021)
• (Jakkala et al., 12 Feb 2026)
• (Wadayama et al., 21 Jun 2026) Determinant-based mutual information denotes a family of dependence functionals in which information is represented through a log-determinant, a determinant ratio, or an operator determinant built from second-order objects such as covariance matrices, conditional error covariances, Gram matrices, or autocorrelation operators. In the literature summarized here, these constructions serve several distinct roles: an alternative mutual-information-like quantity based on covariance log-determinants for blind source separation, a sharp lower bound on Shannon mutual information based on the determinant of conditional-mean prediction error, exact Gaussian-process and linear-Gaussian conditional mutual informations written as log-determinant differences, matrix-based entropy contrasts that recover determinant forms in the $\alpha\to1$ limit, and Fredholm-determinant expressions for continuous random fields (Erdogan, 2022, Bowsher et al., 2014, Skean et al., 2023, Jakkala et al., 12 Feb 2026, Wadayama et al., 21 Jun 2026, Wan et al., 2021).

1. Covariance log-determinants as entropy and mutual information

A central formulation introduces a log-determinant entropy for a random vector $x\in\mathbb R^r$ with covariance $R_x\succ0$. For a small regularization $\epsilon>0$,

$$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$

In the limit $\epsilon\to0$, this coincides with the Gaussian differential entropy $\frac12\log\det R_x+\frac r2\log(2\pi e)$, which upper-bounds the true Shannon entropy of any $x$ with covariance $R_x$ (Erdogan, 2022).

For jointly distributed vectors $x\in\mathbb R^r$ and $x\in\mathbb R^r$0, the same framework defines an LD-conditional entropy through the linear-MMSE error,

$x\in\mathbb R^r$1

and the associated LD-mutual information

$x\in\mathbb R^r$2

Equivalently,

$x\in\mathbb R^r$3

This formulation is explicitly second-order. The corresponding LD-mutual information between two vectors reflects a level of their correlation, rather than arbitrary higher-order statistical dependence. That distinction is operational: the same paper contrasts the LD-infomax criterion with ICA infomax and states that the proposed information maximization approach can separate both dependent and independent sources (Erdogan, 2022).

A different determinant construction appears in regression-based dependence analysis. Let

$x\in\mathbb R^r$4

The scalar dependence measure is

$x\in\mathbb R^r$5

or, in normalized form,

$x\in\mathbb R^r$6

The resulting inequality

$x\in\mathbb R^r$7

is a sharp lower bound on Shannon mutual information, with equality in the jointly Gaussian case (Bowsher et al., 2014).

A plausible summary is that determinant-based mutual information is not a single invariant quantity. In the cited work it includes exact Shannon-MI formulas for Gaussian models, covariance-based surrogates, and lower bounds whose determinant structure is the organizing principle rather than an assertion of universal equivalence.

2. Structural identities, exactness regimes, and common misunderstandings

Several algebraic properties recur across determinant formulations. In the LD framework, $x\in\mathbb R^r$8 for nondegenerate $x\in\mathbb R^r$9, with equality if and only if $R_x\succ0$0. It is symmetric by construction, and if $R_x\succ0$1 is jointly Gaussian then $R_x\succ0$2 equals the Gaussian differential entropy exactly, so $R_x\succ0$3 coincides with Shannon mutual information: $R_x\succ0$4 Thus Gaussian exactness is an explicit property, not a generic one (Erdogan, 2022).

The regression-based bound has an analogous exactness regime. Under the assumptions that $R_x\succ0$5 is one-to-one and continuously differentiable, and that the marginal law of $R_x\succ0$6 is chosen so that $R_x\succ0$7 is Gaussian, one obtains

$R_x\succ0$8

and then

$R_x\succ0$9

When $\epsilon>0$0 is jointly Gaussian, the inequality becomes an equality (Bowsher et al., 2014).

The same determinant structure reappears in exact conditional mutual information for linear Gaussian models. For disjoint index sets $\epsilon>0$1, define

$\epsilon>0$2

and

$\epsilon>0$3

Then

$\epsilon>0$4

This is an exact closed form for jointly Gaussian vectors, obtained directly from Gaussian conditional entropies (Wadayama et al., 21 Jun 2026).

A common misunderstanding is to treat every determinant expression as an exact Shannon mutual information. The cited literature does not support that identification. Exact equality is stated for jointly Gaussian variables, Gaussian processes, linear Gaussian DAGs, and Gaussian random fields under the specified constructions; the regression-error determinant supplies a lower bound; and DiME is described as a lower bound on matrix-based mutual information and as a mutual-information-like quantity rather than as Shannon mutual information itself (Bowsher et al., 2014, Skean et al., 2023, Jakkala et al., 12 Feb 2026, Wadayama et al., 21 Jun 2026, Wan et al., 2021).

3. LD-infomax and determinant maximization in blind source separation

In noiseless blind source separation, one observes mixtures

$\epsilon>0$5

and seeks estimates $\epsilon>0$6 constrained to lie in a known polytope $\epsilon>0$7. The LD-infomax criterion is

$\epsilon>0$8

where the deterministic objective is formed from sample covariances

$\epsilon>0$9

$$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$0

$$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$1

yielding

$$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$2

This gives an information-theoretic perspective for determinant maximization-based structured matrix factorization methods such as nonnegative and polytopic matrix factorization (Erdogan, 2022).

In the limit $$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$3, the noiseless case reduces to

$$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$4

which is the usual determinant-maximization PMF criterion. The reduction is exact in the sense stated in the source summary: the second term is driven to $$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$5 unless $$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$6 and $$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$7 are exactly linearly related (Erdogan, 2022).

The finite-sample perfect-separation guarantee invokes the Polytopic Matrix Factorization identifiability result. If $$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$8 is an identifiable polytope and the true source columns $$H_{LD}^{(\epsilon)}(x) = \frac12\,\log\det\bigl(R_x+\epsilon I_r\bigr) +\frac r2\log(2\pi e).$$9 are sufficiently scattered in the sense that

$\epsilon\to0$0

then any optimizer $\epsilon\to0$1 satisfies

$\epsilon\to0$2

where $\epsilon\to0$3 is a diagonal sign matrix and $\epsilon\to0$4 a permutation. Perfect recovery is therefore guaranteed up to the unavoidable sign-permutation ambiguity from a finite sample $\epsilon\to0$5 that yields a sufficiently scattered configuration (Erdogan, 2022).

4. Regression-error determinants as sharp lower bounds

The regression-based formulation of Bowsher and Voliotis defines dependence through the determinant of the second-moment matrix of the conditional mean prediction error. With

$\epsilon\to0$6

the determinant $\epsilon\to0$7 quantifies the residual uncertainty in $\epsilon\to0$8 after conditioning on $\epsilon\to0$9. The lower bound

$\frac12\log\det R_x+\frac r2\log(2\pi e)$0

is derived by introducing $\frac12\log\det R_x+\frac r2\log(2\pi e)$1, assuming $\frac12\log\det R_x+\frac r2\log(2\pi e)$2 is an invertible, continuously differentiable transform of $\frac12\log\det R_x+\frac r2\log(2\pi e)$3 with Gaussian marginal law, and then applying a covariance-determinant bound to $\frac12\log\det R_x+\frac r2\log(2\pi e)$4 (Bowsher et al., 2014).

The derivation uses the identity

$\frac12\log\det R_x+\frac r2\log(2\pi e)$5

which implies

$\frac12\log\det R_x+\frac r2\log(2\pi e)$6

Consequently, the determinant in the bound is the determinant of the usual law-of-total-variance residual covariance. The paper further states that the bound is tighter than lower bounds based on the Pearson correlation and ones derived using average mean square-error rate distortion arguments (Bowsher et al., 2014).

The comparison is explicit in the bivariate case: $\frac12\log\det R_x+\frac r2\log(2\pi e)$7 Because $\frac12\log\det R_x+\frac r2\log(2\pi e)$8, the determinant-based bound is at least as large, and strictly larger whenever $\frac12\log\det R_x+\frac r2\log(2\pi e)$9 fails to capture non-linear or higher-order dependence. The same summary states that the determinant-based bound is strictly tighter than the average-MSE bound by an AM-GM inequality on eigenvalues (Bowsher et al., 2014).

Estimation proceeds by nonparametric regression: estimate $x$0, form the empirical second-moment matrix of residuals,

$x$1

and compute $x$2, with plug-in lower bound

$x$3

The method includes BC$x$4 bootstrap intervals for $x$5 and a composite estimator

$x$6

which substantially improves upon inference about mutual information based on $x$7-nearest neighbour estimators alone in the simulations described in the source summary (Bowsher et al., 2014).

5. Matrix-based entropies, Gram determinants, and DiME

A distinct line of work defines matrix-based Rényi entropies from normalized Gram matrices. Given samples $x$8 and a positive-definite kernel $x$9 with $R_x$0, form the Gram matrix $R_x$1 and its eigenvalues $R_x$2. The $R_x$3-order matrix-based Rényi entropy is

$R_x$4

For the trace-normalized matrix $R_x$5, one has $R_x$6. In the limit $R_x$7,

$R_x$8

and, up to additive constants,

$R_x$9

for small $x\in\mathbb R^r$0, recovering the familiar log-determinant formula from Gaussian information (Skean et al., 2023).

For two views with kernels $x\in\mathbb R^r$1 and $x\in\mathbb R^r$2, the joint Gram matrix is defined by the Hadamard product

$x\in\mathbb R^r$3

and the matrix-based mutual information of order $x\in\mathbb R^r$4 is

$x\in\mathbb R^r$5

In the $x\in\mathbb R^r$6 limit this recovers a log-determinant form analogous to

$x\in\mathbb R^r$7

DiME is defined by contrasting the joint entropy of paired samples with that of negatively paired samples obtained by permuting one view: $x\in\mathbb R^r$8

$x\in\mathbb R^r$9

The same source states that this simpler form shows that DiME is a lower bound on $x\in\mathbb R^r$00. It also gives a collapse-avoidance property: if one view collapses so that $x\in\mathbb R^r$01, then $x\in\mathbb R^r$02 and $x\in\mathbb R^r$03, so any collapse yields zero objective (Skean et al., 2023).

Finite-sample computation is spectral: choose kernels, compute Gram matrices, normalize, form $x\in\mathbb R^r$04, diagonalize $x\in\mathbb R^r$05 and permuted copies, evaluate $x\in\mathbb R^r$06, and average over a small set of random permutations. A single eigendecomposition costs $x\in\mathbb R^r$07, low-rank approximations such as Nyström or random features reduce this to $x\in\mathbb R^r$08 with $x\in\mathbb R^r$09, and the gradient is obtained through

$x\in\mathbb R^r$10

followed by the chain rule. The paper reports use cases in multiview representation learning and latent factor disentanglement, and compares DiME against InfoNCE, NWJ, JS, CLUB, MINE, CKA, HSIC, TUBA, DoE, and plain matrix-based MI in the specific experiments summarized in the source text (Skean et al., 2023).

6. Gaussian-process and linear-Gaussian network formulations

In Gaussian-process information gathering, determinant-based mutual information is exact. For a finite candidate set $x\in\mathbb R^r$11 with prior covariance $x\in\mathbb R^r$12, and a selected subset $x\in\mathbb R^r$13, the MI between noisy observations at $x\in\mathbb R^r$14 and the rest of the field is

$x\in\mathbb R^r$15

Equivalently,

$x\in\mathbb R^r$16

The standard exact evaluation costs $x\in\mathbb R^r$17 per determinant of size $x\in\mathbb R^r$18 (Jakkala et al., 12 Feb 2026).

Schur-MI exploits the iterative structure of robotic information gathering and the Schur-complement determinant identity

$x\in\mathbb R^r$19

With the conditional covariance

$x\in\mathbb R^r$20

one obtains

$x\in\mathbb R^r$21

After precomputing $x\in\mathbb R^r$22, each evaluation requires only two $x\in\mathbb R^r$23 determinants, reducing the per-evaluation cost from $x\in\mathbb R^r$24 to $x\in\mathbb R^r$25. The paper states that MI is submodular, so the greedy algorithm achieves a $x\in\mathbb R^r$26 approximation to the optimal NP-hard sensor placement problem, and reports up to a $x\in\mathbb R^r$27 speedup over the standard log-det formulation while producing identical MI values (Jakkala et al., 12 Feb 2026).

For multi-terminal linear Gaussian wireless networks, conditional mutual information is also a log-determinant difference of block Schur complements: $x\in\mathbb R^r$28 The node-pair covariances are produced by a topologically ordered K-recursion,

$x\in\mathbb R^r$29

after which all computations are expressed through automatic-differentiation primitives: matrix products, Hermitian transposes, Cholesky decompositions, triangular solves, and log-determinants. A single reverse-mode sweep yields the Wirtinger gradient with respect to all controllable factors at once, allowing projected gradient iterations for weighted sum-rate, secrecy, and non-linear composites built from finitely many conditional MIs (Wadayama et al., 21 Jun 2026).

7. Random fields, Fredholm determinants, and spatial spectra

For continuous electromagnetic fields, determinant-based mutual information is formulated at the operator level. A current density $x\in\mathbb R^r$30 on a source region $x\in\mathbb R^r$31 generates an electric field

$x\in\mathbb R^r$32

and the receiver observes

$x\in\mathbb R^r$33

with $x\in\mathbb R^r$34 and $x\in\mathbb R^r$35 modeled as zero-mean Gaussian random fields. Their second moments define signal and noise autocorrelation operators $x\in\mathbb R^r$36 and $x\in\mathbb R^r$37 on $x\in\mathbb R^r$38 (Wan et al., 2021).

When the field autocorrelation kernel is continuous or Hilbert-Schmidt, Mercer’s theorem yields eigenfunctions $x\in\mathbb R^r$39 and eigenvalues $x\in\mathbb R^r$40 such that

$x\in\mathbb R^r$41

and

$x\in\mathbb R^r$42

Under spatially white Gaussian noise $x\in\mathbb R^r$43, the mutual information per channel use becomes

$x\in\mathbb R^r$44

equivalently

$x\in\mathbb R^r$45

This is a Fredholm determinant expression for a continuous-field Gaussian channel (Wan et al., 2021).

The same source gives two extensions. First, for rational-spectrum kernels one may define

$x\in\mathbb R^r$46

and derive analytic closed forms without listing the individual eigenvalues. For the one-dimensional exponential kernel

$x\in\mathbb R^r$47

the mutual information is

$x\in\mathbb R^r$48

Second, for colored noise one forms

$x\in\mathbb R^r$49

whose generalized eigenvalues $x\in\mathbb R^r$50 yield

$x\in\mathbb R^r$51

In the stationary infinite-region limit, the operator determinant becomes the spatial spectral integral

$x\in\mathbb R^r$52

with water-filling

$x\in\mathbb R^r$53

This is presented as the spatial-domain analog of Shannon’s classical continuous-time AWGN result (Wan et al., 2021).

Across these formulations, the determinant plays a unifying but not uniform role. In covariance-based LD entropy it defines a mutual-information-like dependence tied to second-order structure; in regression it controls a sharp lower bound through conditional prediction error; in Gaussian-process, Gaussian-DAG, and random-field models it gives exact mutual information or conditional mutual information; and in matrix-based entropy methods it reappears as the $x\in\mathbb R^r$54 limit of spectral entropy constructions that are optimized directly from data (Erdogan, 2022, Bowsher et al., 2014, Skean et al., 2023, Jakkala et al., 12 Feb 2026, Wadayama et al., 21 Jun 2026, Wan et al., 2021).