Law of Information Capacity

• The law of information capacity is a principle that defines the maximum extractable information from any physical channel based on energy, time, and quantum limits.
• It unifies classical communications, quantum theory, and thermodynamics by establishing universal bounds through metrics like the Holevo quantity and Shannon–Hartley law.
• Practical applications include optimizing channel design and coding strategies across systems from silicon photonics to complex network infrastructures.

The Law of Information Capacity formalizes fundamental, often universal constraints on the maximum information that can be transmitted, processed, or extracted through a physical system or channel. It delineates the ultimate limits—arising from energetic, temporal, geometric, probabilistic, or quantum-mechanical considerations—on mutual information, channel capacity, or related operational measures, independent of implementation details. Precise instantiations of the law reveal deep connections among classical communication theory, quantum information, thermodynamics, network dynamics, and field theory.

1. Foundational Principles and Universal Bounds

At its core, the Law of Information Capacity asserts the existence of a supremum—often expressible in closed form—on the transmittable information in a channel characterized by given physical resources and constraints. A canonical example is the universal bound derived from the Holevo quantity in quantum field-theoretic settings:

$$H(A\!:\!B) \leq \chi \leq \frac{E \, \Delta t}{\hbar}$$

where $E$ is the average signal energy, $\Delta t$ the operational detection time, and $\hbar$ Planck’s constant (Bousso, 2016). This bound applies regardless of the size, mass, or quantum/classical nature of the communicating systems, imposing an upper limit on accessible information based solely on the energy-time resources.

In silicon photonics and other nonlinear optical waveguides, information capacity is similarly bounded not merely by linear noise but also by nonclassical, multiplicative loss fluctuations such as two-photon absorption (TPA) and free-carrier absorption (FCA) (Dimitropoulos et al., 2014). Here, the peak per-Hz capacity saturates (9–11 bits/s·Hz) due to power-dependent noise, with optimal launch powers constrained to ~0.1–1 mW.

In classical and quantum statistical mechanics, there is a rigorous equivalence between capacity maximization and free energy minimization: maximizing mutual information is thermodynamically isomorphic to extracting maximal work, a manifestation of the capacity–free-energy duality (0809.3540).

2. Mathematical Formulations

The Law of Information Capacity is instantiated through a variety of mathematical forms according to the operational regime and physical context.

Shannon–Hartley Law for AWGN channels:

$C = W \log_2\left(1 + \frac{P}{N_0 W}\right)$

where $W$ is channel bandwidth, $P$ input power, and $N_0$ noise spectral density (Chen, 2016, 0901.4420).

Quantum and field-theoretic universal bound:

$H(A\!:\!B) \leq \chi \leq \frac{E \Delta t}{\hbar}$

with $\chi$ the Holevo quantity as the maximal extractable classical information in a quantum setting (Bousso, 2016).

Bandlimited signal models (Kolmogorov $\epsilon$-entropy/capacity):

$\bar{C} \sim N_0 \log \sqrt{\text{SNR}}$

with $N_0$ the number of degrees of freedom and SNR the relevant signal-to-noise ratio (Franceschetti et al., 2015).

Weak-signal regime (Fisher information):

$$I(\Theta ; R) \approx \frac{1}{2}\mathrm{Tr}\left( J C_\Theta \right)$$

with $J$ the Fisher information matrix and $C_\Theta$ the input covariance at vanishing power (Kostal, 2010).

Generalized quantum theory and GPT channel capacity:

$$C^\epsilon(\mathcal{N}) \leq \sup_{P_X} D_H^\epsilon(\pi^{AB}_{P_X} \| \pi^A_{P_X} \otimes \pi^B_{P_X})$$

where $D_H^\epsilon$ is the hypothesis-testing relative entropy (Minagawa et al., 2023).

3. Operational and Physical Interpretations

The law impacts practical channel design, coding theory, and the delineation of quantum advantage:

• Energy–Time Limits: There is a strict upper bound on the amount of information that can be conveyed with finite energy over finite time, independent of domain (optical, electronic, or relativistic communications) (Bousso, 2016).
• Multiplicative Noise and Nonlinear Effects: In materials like silicon, nonlinear processes fundamentally cap achievable information rates, a feature absent in ideal linear Gaussian noise models (Dimitropoulos et al., 2014).
• Thermodynamic Duals: In statistical mechanics, capacity optimization is dual to thermodynamic minimization of Gibbs free energy, unifying work extraction and information transmission paradigms (0809.3540).

4. Extensions Across Theoretical Frameworks

The Law of Information Capacity is robust under generalizations:

• General Physical Theories (GPTs): Capacity, both one-shot and asymptotic, is governed by operationally defined hypothesis-testing relative entropy, yielding universal expressions for channel performance independent of classical/quantum structure (Minagawa et al., 2023).
• Field Theory and Fisher Information: In the EPI formalism, the channel capacity (Fisher information trace) directly yields the kinetic action in Lagrangian field theory. Extremizing this sum, subject to structural constraints, reproduces canonical field equations (Sładkowski et al., 2012).
• Generalized Transform Domains: For AWGN channels, changing the signal domain (Fourier, fractional, LCT, etc.) leads to generalized capacity laws, where capacity depends on transformed bandwidth and SNR per transformed degree of freedom (0901.4420).

5. Network, Nonlinear, and Deterministic Regimes

In networked systems with hard node capacity limits, the law takes on a dynamic facet:

• Diminishing Marginal Returns: Increasing input past a node’s capacity produces sublinear increases in output (i.e., $dF(u)/du$ is monotonically decreasing), resulting in exponential incoming and fat-tailed outgoing information flow distributions in complex networks (Marinazzo et al., 2012).
• Deterministic Channels: Even in zero-noise or bounded-noise deterministic systems, capacity per degree of freedom grows logarithmically in SNR, mirroring stochastic laws (Franceschetti et al., 2015).
• Receiver Side Information and Outage: For channels with variable state and decoder side information, capacity laws are parameterized by information density tails, allowing definitions such as capacity versus outage and expected capacity (0804.4239).

6. Quantum Information Constraints and Conservation

Quantum channels exhibit unique conservation constraints:

• Information Conservation Law: The sum of mutual informations between a reference and each output subsystem in a bipartite isometric channel is strictly conserved, yielding nonclassical bounds on randomness cost and masking ability of quantum channels (Lie et al., 2019).
• Generalized Information Quantities: Recent formulations introduce generalized information quantities unifying classical mutual information and coherent information into a single operationally meaningful object that governs channel capacity for mixed quantum-classical sources (Khanian, 2023).

7. Implications and Unifying Perspective

The Law of Information Capacity, in all its forms, imposes non-negotiable, physically rooted limitations on all communication, estimation, and processing scenarios. Whether formulated in terms of energy-time, SNR, Fisher information, quantum entropy, or general operational metrics, it constrains channel design, error correction, and even physical law (as in action principles for field theories). The universality of these bounds reflects structural properties of physics and information theory, unifying distinct disciplines under the common imperative that information flow cannot exceed what is permitted by available resources, channel structure, and fundamental physical principles (Bousso, 2016, Dimitropoulos et al., 2014, Minagawa et al., 2023, 0809.3540, Jeong et al., 2018, Franceschetti et al., 2015, Sładkowski et al., 2012, Kostal, 2010).