Entropy Principle in Physics & Mathematics

• Entropy principle is a family of variational and extremal statements defining systems via maximum entropy, second-law behavior, and dynamical complexity.
• It unifies methodologies from statistical mechanics, quantum theory, and gravitation using constrained optimization and multiplicity arguments.
• The approach informs practical applications from uncertain data inference and quantum channel characterization to recovering gravitational potentials through thermodynamic extremization.

The expression entropy principle denotes several distinct but structurally related ideas across statistical mechanics, thermodynamics, quantum theory, gravitation, and dynamical systems. In some contexts it refers to the maximum entropy principle, where an admissible state or process is selected by entropy maximization under constraints. In others it denotes a second-law-type principle, where entropy production or entropy increase is characterized operationally, often relative to coarse-graining, measurement, or incomplete information. In dynamical systems, the same expression often denotes a variational principle equating topological entropy with a supremum of measure-theoretic entropies. The modern literature therefore treats the entropy principle less as a single theorem than as a family of extremal and variational statements whose precise content depends on the underlying state space, observables, and admissible constraints (Lee, 2012, Varizi et al., 2024, Roupas, 2020, Kerr et al., 2010).

1. Maximum entropy, multiplicity, and statistical-mechanical foundations

In equilibrium statistical mechanics, the classical form of the entropy principle is the maximum entropy principle associated with Gibbs–Shannon or von Neumann entropy. One standard formulation maximizes

$H[p]=-\sum_i p_i\ln p_i$

subject to normalization and a mean-energy constraint, yielding the Boltzmann distribution. A rigorous bridge between this open-system variational principle and the microcanonical description of a larger closed universe was given by showing that the open-system MaxEnt functional arises by partial maximization of the Gibbs–Shannon entropy of the closed universe over heat-bath degrees of freedom. In that derivation, the canonical objective

$H[p]-\beta\sum_i p_iE_i$

is not an independent postulate but the reduced form of the microcanonical entropy maximization problem for system plus bath (Lee, 2012).

A complementary line of work derives entropy directly from multiplicity. For independent multinomial processes, histogram probabilities factor into a multiplicity term and a bias term, and the scaled logarithm of multiplicity gives the Boltzmann–Gibbs–Shannon entropy. The same paper argues that once independence is relaxed, the admissible entropies compatible with the first three Shannon–Khinchin axioms are the $(c,d)$-entropies, and that a generalized maximum entropy principle remains meaningful for non-ergodic and complex systems whenever the corresponding relative entropy can still be factored into a generalized multiplicity and a constraint term (Hanel et al., 2014).

The status of constrained maximization itself has also been reinterpreted. One proposal argues that the usual Gibbs/exponential family need not be derived from a literal maximization principle if one instead assumes the existence of a phenomenological entropy function $\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$ consistent with the microscopic entropy functional and stable under infinitesimal changes of the underlying probability density. Under that weaker assumption, the same exponential family follows uniquely, with

$$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$

so the entropy principle becomes a consistency principle between microscopic and phenomenological descriptions rather than an explicit constrained optimization rule (Rudnicki, 2014).

2. Extensions under incomplete information and at the level of processes

A substantial recent extension of the maximum entropy principle concerns uncertain or partially observed data. In this setting, one does not observe the model variable $X$ directly, but only an observation variable $O$ through an observation model $P(O\mid X)$. The resulting principle of uncertain maximum entropy replaces direct moment matching by posterior-averaged constraints of the form

$$\sum_{x\in X}\Pr(x)\phi_k(x)=\sum_{o\in O}\tilde{\Pr}(o)\sum_{x\in X}\Pr(x\mid o)\phi_k(x),$$

so the empirical side of the constraint is itself model-dependent. This framework generalizes both ordinary maximum entropy and latent maximum entropy, and the proposed solution strategy is an expectation-maximization construction in which the E-step computes posterior feature expectations and the M-step solves a standard convex MaxEnt problem with those expected sufficient statistics as targets (Bogert, 2021, Bogert et al., 2023).

The same literature also develops a practical algorithmic implementation, denoted \texttt{uMaxEnt}, together with two comparison baselines, \texttt{Most-Likely-x} and \texttt{MaxEnt-MaxEnt}. The stated motivation is interpretive as well as computational: if observations are noisy, ambiguous, or partial, then ad hoc relaxation of exact MaxEnt constraints weakens the usual “least biased distribution consistent with the evidence” reading, whereas uncertain maximum entropy keeps the uncertainty in the constraints themselves (Bogert et al., 2023).

An independent generalization lifts the entropy principle from states to quantum channels. For a channel $\mathcal N_{A'\to A}$, the paper defines a channel entropy

$H[p]-\beta\sum_i p_iE_i$0

and a channel mean energy

$H[p]-\beta\sum_i p_iE_i$1

The corresponding maximum entropy principle for quantum processes states that

$H[p]-\beta\sum_i p_iE_i$2

and that the maximizer is unique: it is the absolutely thermalizing channel $H[p]-\beta\sum_i p_iE_i$3, the replacer channel that outputs the Gibbs state $H[p]-\beta\sum_i p_iE_i$4 for every input (Das et al., 30 Jun 2025).

The same variational logic has also been used in a high-energy application. Treating the Higgs branching ratios $H[p]-\beta\sum_i p_iE_i$5 as a probability distribution over mutually exclusive decay channels, the entropy of the multinomial decay process for a large ensemble of Higgs bosons reduces asymptotically to

$H[p]-\beta\sum_i p_iE_i$6

Maximizing this quantity with respect to $H[p]-\beta\sum_i p_iE_i$7 yields

$H[p]-\beta\sum_i p_iE_i$8

and the same formalism is then used to study a Higgs sector with an additional invisible decay channel (Alves et al., 2014).

3. Quantum entropy principles, measurement, and the second law

In quantum thermodynamics, one influential refinement of the entropy principle replaces the vague claim that entropy always increases by a precise statement about reduced dynamics. If the reduced evolution of a subsystem is described by a trace-preserving completely positive map $H[p]-\beta\sum_i p_iE_i$9, then

$(c,d)$0

where $(c,d)$1 is the von Neumann entropy. The paper emphasizes that, in finite dimensions, unitality is not only sufficient but also necessary for a general non-diminishing entropy statement. Cooling and Maxwell-demon-type operations are therefore associated with non-unital channels, while heating can arise from unital ones; the same global unitary can induce opposite entropy behavior on different subsystems (Kirsanov et al., 2018).

A broader operational characterization appears in the so-called catalytic entropy principles. For a state $(c,d)$2, the von Neumann entropy is identified with the minimum entropy obtainable after dephasing: $(c,d)$3 and, for a purification $(c,d)$4, also with the maximum classical mutual information obtainable from local measurements,

$(c,d)$5

The same scheme is extended to Rényi, Tsallis, and generalized spectral entropies, and is then combined with catalyst-assisted state-conversion and cooling statements (Luo et al., 2021).

Another quantum entropy principle takes the form of an entropic uncertainty relation. For a density matrix $(c,d)$6, with diagonal distributions $(c,d)$7 and $(c,d)$8 in two bases $(c,d)$9 and $\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$0, Frank and Lieb show

$\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$1

where $\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$2. In this setting the entropy principle is an uncertainty principle: the sum of the classical entropies of the diagonals in two representations is bounded below by the von Neumann entropy and a basis-overlap term (Frank et al., 2011).

Classical continuum thermodynamics gives yet another formulation. In Rational Extended Thermodynamics, the entropy principle combines the existence of an entropy balance law implied by the original balance system, the sign condition $\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$3, and concavity of the entropy density $\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$4. The paper reformulates the structural part in terms of the vector space of supplementary balance laws, derives the associated Lagrange–Liu equations, and identifies an overdetermined second-order PDE system whose solutions generate all supplementary balance laws, with entropy as the distinguished member selected by concavity and nonnegative production (Preston, 2010).

The status of the second law as a universal monotonicity principle has also been challenged. One paper argues that, under time-reversal invariant microscopic dynamics and a time-reversal symmetric coarse-grained entropy assignment $\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$5, a universal statement of the form “entropy does not decrease” is inconsistent. Its “mirror-state paradox” shows that applying such a law to a trajectory and to its time reverse forces every time to be a local minimum, which in turn makes entropy constant. The proposed replacement is explicitly distributional: entropy should be treated as a stochastic variable with a time-dependent or long-time distribution $\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$6 or $\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$7, reshaped by constraints and boundary conditions rather than endowed with a universal direction (Peng, 17 Feb 2026).

4. Entropy production and maximum-entropy inference

A unifying treatment of entropy production follows directly from Jaynes’ maximum entropy principle. Given incomplete access to an input state $\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$8 and possibly to expectation values after a quantum channel $\mathcal S(F_1,\dots,F_M;\{\mathcal V\})$9, one constructs the maximum-entropy state

$$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$0

consistent with the accessible data. Entropy production is then defined as the quantum relative entropy

$$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$1

This makes entropy production explicitly dependent on the observer’s accessible information. Under fine-grained projective measurement it reduces to diagonal entropy minus von Neumann entropy; under coarse-grained projective measurement it becomes observational entropy minus von Neumann entropy; for an open system with inaccessible environment it reproduces the standard information-theoretic entropy production

$$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$2

together with the decomposition into environment mismatch and system–environment mutual information (Varizi et al., 2024).

A different route to entropy production dynamics uses the speed-gradient principle. For a time-dependent pdf $$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$3 on a compact carrier $$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$4, the entropy

$$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$5

is treated as a control objective, and the induced evolution law is a projected steepest-ascent dynamics of the form

$$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$6

Under normalization alone, the unique asymptotic limit is the uniform density $$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$7; with normalization and conserved energy, the limit is the Gibbs form $$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$8. This is રજૂced as a dynamical justification of a maximum entropy production principle: MaxEnt determines the endpoint, while the speed-gradient construction determines the path of maximal instantaneous entropy increase compatible with the constraints (Fradkov et al., 2014).

5. Entropy extremization in gravitation

In general relativity, the entropy principle has been used to reconstruct part of the gravitational field itself. For a static, spherically symmetric spacetime

$$\beta_j=\frac{1}{k_B}\frac{\partial \mathcal S}{\partial F_j},$$9

the Hamiltonian constraint fixes the spatial metric component $X$0 through

$X$1

The paper then extremizes the total entropy of a generic thermodynamic medium over the matter-filled region at fixed total mass-energy and fixed particle numbers, using only local equilibrium thermodynamics and the Hamiltonian constraint. The resulting variational calculation yields $X$2, a differential equation for the temperature profile, and—by reading off the integrating factor—the redshift factor $X$3. The final relation is exactly Tolman’s law,

$X$4

and, after substitution of the Hamiltonian-constraint form of $X$5, the standard interior general-relativistic redshift potential is recovered without separately assuming the continuity equation, the Tolman relation, or the TOV equation (Roupas, 2020).

In this formulation, the gravitational potential

$X$6

emerges as the thermodynamic integrating factor associated with entropy maximization. In the weak-field, nonrelativistic limit it reduces to the Newtonian potential, so the same construction yields both the relativistic redshift factor and the ordinary gravitational potential. The result is explicitly limited to static, spherically symmetric spacetimes with matter in local thermodynamic equilibrium and asymptotic normalization $X$7 (Roupas, 2020).

6. Variational principles in dynamical systems

In dynamical systems, the phrase entropy principle typically refers not to entropy maximization of states, but to a variational principle equating topological entropy with a supremum of measure-theoretic entropies. For actions of sofic groups on compact metrizable spaces, Kerr and Li established

$X$8

using an operator-algebraic framework based on approximately equivariant maps into finite-dimensional commutative $X$9-algebras. This replaced the earlier finite-partition approach by a function-based formalism and produced both measure and topological sofic entropy in a common language (Kerr et al., 2010).

Zhang localized this result to finite open covers. For a countable sofic group action $O$0 and a finite open cover $O$1, the local variational principle is

$O$2

with the right-hand side interpreted as $O$3 if $O$4. This local form is then used to analyze entropy tuples and to connect localized sofic entropy with its classical amenable-group counterpart (Zhang, 2011).

For locally compact separable metrizable systems, the classical compact variational principle was extended from proper maps to arbitrary continuous maps. With topological entropy defined via admissible covers and Bowen entropy minimized over compatible metrics, the resulting identity is

$O$5

A notable corollary is that any linear transformation $O$6 on a finite-dimensional vector space has null topological entropy (Caldas et al., 2015).

A related random version has also been established for continuous bundle random dynamical systems over an infinite countable discrete amenable group. There the topological entropy is defined by fiberwise separated sets,

$O$7

and the variational principle becomes

$O$8

with restriction to ergodic measures when the base system is ergodic (Lian, 2024).

Taken together, these results show that the entropy principle in mathematics usually means an equality between topological and measure-theoretic complexity, whereas in thermodynamics and quantum theory it more often means an extremal or monotonicity principle for states, processes, or coarse-grained descriptions. The common structure is variational: entropy functions either select admissible objects or provide the exact bridge between microscopic and macroscopic descriptions.