Logarithmic Derivative of Mutual Information

• Logarithmic derivative of mutual information is defined as the derivative of ln I(θ) with respect to a scaling parameter, capturing how information scales in complex systems.
• In quantum field theory, it serves as a diagnostic for renormalizability, distinguishing super-renormalizable, marginal, and non-renormalizable interactions via its decay or growth rates.
• In communication models, it generalizes the I-MMSE identity, linking channel quality with estimation efficacy and providing insights into asymptotic scaling laws.

The logarithmic derivative of mutual information quantifies the rate of change of the logarithm of mutual information with respect to a characteristic scale or parameter, and serves as a sensitive probe of scaling behavior, information transmission, and renormalization structure in a range of physical and information-theoretic settings. Its rigorous analysis has emerged in field theory, non-linear channels, and the information theory of noisy communications, providing a unified diagnostic for scale invariance, estimation efficacy, and UV–IR correlation decay.

1. Formal Definitions and General Properties

The logarithmic derivative of mutual information is defined as the derivative of $\ln I(\theta)$ with respect to a parameter $\theta$ that controls the scale, quality, or separation in the system: $$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$ The precise meaning of $\theta$ is context-dependent:

• In quantum field theory (QFT), $\theta$ may be the logarithm of the ratio of momentum scales, $r = B/A$.
• In information theory, $\theta$ can represent the signal-to-noise ratio (SNR) or a generalized channel quality parameter.

The logarithmic derivative captures the scaling exponent of mutual information with respect to the parameter of interest. If $I(\theta)\sim \theta^\alpha$ as $\theta\to\infty$, then $\frac{d}{d\ln \theta} \ln I(\theta)\to\alpha$.

2. Quantum Field Theory: Diagnostic for Renormalizability

In quantum field theory, the central setup considers the mutual information between two infinitesimal, non-overlapping shells $\theta$0 and $\theta$1 in momentum space, separated by a ratio $\theta$2. For a global pure state $\theta$3 of the field, the mutual information density is

$\theta$4

where $\theta$5 is the volume factor. The key diagnostic is the large-mode-separation logarithmic slope: $\theta$6 For the Klein–Gordon field with local self-interaction of the form $\theta$7, analytical and perturbative computations yield: $\theta$8 where $\theta$9 is the canonical mass dimension of the coupling $$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$0,

$$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$1

The physical content is summarized in the following classification:

Theory type	$$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$2	$$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$3 behavior
Super-renormalizable	$$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$4	$$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$5 (decaying)
Marginal (renormalizable)	$$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$6	$$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$7 (saturating)
Non-renormalizable	$$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$8	$$\frac{d}{d\theta}\ln I(\theta) = \frac{1}{I(\theta)}\frac{dI(\theta)}{d\theta}.$$9 (growing)

A negative slope corresponds to rapidly decaying ultraviolet–infrared (UV–IR) correlations (super-renormalizable), zero slope to scale-invariance (marginal), and positive slope to growing UV–IR mutual information, indicating strong non-renormalizable (irrelevant) interactions. This scaling holds for both equilibrium and out-of-equilibrium systems, as verified in Minkowski and conformal de Sitter spacetimes for various self-interaction terms. The universality of $\theta$0 as a renormalization diagnostic is substantiated at leading order in perturbation theory and requires appropriate normal-ordering subtractions for finiteness of correlation functions (Bowen et al., 12 Nov 2025).

3. Nonlinear and Classical Channels: Gamma and Gaussian Cases

For channel models in information theory, the logarithmic derivative tracks how information transmission varies with channel quality or noise. In the non-linear gamma channel,

$\theta$1

with $\theta$2, $\theta$3, and $\theta$4 (input). The mutual information $\theta$5 depends on the channel parameter $\theta$6. The exact logarithmic derivative is (Arras et al., 2016): $\theta$7 where $\theta$8. This formula generalizes the classical I-MMSE identity for Gaussian noise channels, where

$\theta$9

with $\theta$0 the minimum mean-squared error for observation $\theta$1. The gamma-channel case lacks a single MMSE-like term; instead, the difference of two conditional expectation squares reflects the non-linearity of the channel.

4. Scaling Laws and Asymptotic Regimes

The logarithmic derivative reveals universal asymptotic regimes:

• Large Separation/High Channel Quality:
  • In the QFT context, for $\theta$2, $\theta$3 directly measures the decay or persistence of UV–IR mutual information.
  • For the gamma channel, $\theta$4 at large $\theta$5, so the logarithmic derivative approaches $\theta$6.
  • For Gaussian channels with Gaussian input, $\theta$7 yields a logarithmic derivative of $\theta$8.
• Small Parameter Expansion:
  • For the gamma channel, $\theta$9 as $r = B/A$0, so the logarithmic derivative starts linear in $r = B/A$1.
  • In Gaussian channels, the small-SNR limit yields $r = B/A$2.

Upper and lower bounds on $r = B/A$3 are sharp in important cases: for $r = B/A$4,

$r = B/A$5

5. Connections to Information Theory, Estimation, and Cumulant Structures

The logarithmic derivative’s information-theoretic significance is grounded in its links to estimation theory and moment-cumulant analogies:

• For Gaussian channels, the first and higher derivatives of mutual information with respect to SNR have a "cumulant-moment" structure, with the logarithmic derivative at leading order proportional to MMSE and higher derivatives involving combinatorial forms in the conditional central moments $r = B/A$6 (Nguyen, 2023).
• For general channels, the logarithmic derivative encodes how efficiently information is transmitted as the underlying channel is tuned.
• In quantum field theory, the analog is the degree to which correlations between degrees of freedom at disparate momentum scales persist under the RG flow, directly tying to physical renormalizability.

6. Limitations and Technical Conditions

Results for the logarithmic derivative of mutual information are typically established under the following technical conditions:

• In QFT, perturbation theory to second order in the coupling and appropriate normal-ordering subtractions to control divergences (Bowen et al., 12 Nov 2025).
• For the gamma channel, finiteness of higher moments ($r = B/A$7) and positivity of input.
• In information theory models, differentiability of the mutual information and appropriate regularity of input/output distributions.

This suggests the general applicability of the logarithmic derivative as a diagnostic is contingent on the differentiability and analytic behavior of mutual information under parameter variation, and care is required when strong-coupling or non-analytic effects occur.

7. Synthesis and Significance

The logarithmic derivative of mutual information provides a universal, quantitative diagnostic for the scaling and structure of correlations in complex systems:

• In field theory, it is a robust and direct indicator of renormalizability—negative for super-renormalizable, zero for marginal, and positive for non-renormalizable interactions.
• In estimation theory, it measures the information increase per incremental improvement in channel quality, and is tightly bound to the MMSE in the Gaussian case or its analogues in more complicated channel families.
• Across contexts, it encapsulates how inter-scale or input-output dependencies decay or persist, offering a unifying framework for analyzing the flow of information in both physical and information-theoretic systems.