Geometry of Rényi Entropy on the Majorization Lattice

Abstract: Majorization is a stochastic ordering relation that compares the relative diversity of probability distributions with numerous applications in econometrics, spectral theory, and ecology. It is well-known that the majorization partial order forms a complete lattice on the set of ordered probability distributions. In this work, we study the properties of Rényi entropy on the majorization lattice. We establish a fundamental relation between the comonotone coupling and the independent coupling associated with a collection of marginal distributions. Consequently, we show that, for every order $ α\in [0,\infty] $, the Rényi entropy is subadditive on the majorization lattice. We further characterize the supermodular regime, showing that Rényi entropy is supermodular on the majorization lattice for $ α\in {0} \cup [1,\infty] $.

• The paper establishes subadditivity of Rényi entropy on the majorization lattice, with equality only for deterministic PMFs, thereby extending classical Shannon entropy results.
• It characterizes the supermodular regime for Rényi entropy, proving valid inequalities for orders {0} ∪ [1, ∞] and identifying failures for orders in (0, 1) through numerical examples.
• The study introduces a Rényi-parametrized distance metric based on couplings and Lorenz curves, linking theoretical findings with applications in econometrics, quantum information, and statistical modeling.

Structural Properties of Rényi Entropy on the Majorization Lattice

Introduction

The paper "Geometry of Rényi Entropy on the Majorization Lattice" (2605.09655) investigates the structural characteristics of Rényi entropy in the context of majorization lattices. Majorization provides a framework for comparing the dispersion of probability distributions, with widespread relevance across fields such as econometrics, quantum information, spectral theory, and statistics. The study extends prior results on Shannon entropy by Cicalese and Vaccaro, now encompassing the entire family of Rényi entropies. The approach leverages the geometry of couplings and Lorenz curves to derive results with strong structural and numerical rigor.

Majorization Lattice and Coupling Structures

Majorization is a partial order on the set of ordered probability mass functions (PMFs), defined via partial sums comparison. When PMFs are sorted in non-increasing order, the majorization relation forms a complete lattice. The paper constructs explicit glb (greatest lower bound) and lub (least upper bound) elements for any pair of PMFs based on comonotone (north-west) and independent couplings.

A central technical finding is that comonotone couplings always majorize independent couplings for any pair (or collection) of marginal PMFs. This relationship is foundational, enabling tractable comparisons and tight bounds between entropy values resulting from joint distributions constructed by coupling.

Subadditivity of Rényi Entropy

The core property established is the subadditivity of Rényi entropy over the majorization lattice for every order $\alpha \in [0, \infty]$:

$$H_\alpha(p \wedge q) \leq H_\alpha(p) + H_\alpha(q)$$

with equality if and only if at least one PMF is deterministic. The proof builds on the structural comparison between couplings and the Schur-concavity of Rényi entropy. Notably, the result generalizes to collections of PMFs: the Rényi entropy of the glb of any finite set is bounded by the sum of their individual entropies. Furthermore, an explicit upper bound is derived:

$$H_\alpha(p) + H_\alpha(q) \leq 2 H_\alpha(p \wedge q)$$

with equality realized exclusively when $p=q$ for $\alpha \in (0, \infty)$.

Supermodularity of Rényi Entropy

Another principal contribution is the characterization of the supermodular regime:

$$H_\alpha(p) + H_\alpha(q) \leq H_\alpha(p \wedge q) + H_\alpha(p \vee q)$$

for $\alpha \in \{0\} \cup [1, \infty]$. The property holds with equality for edge cases $\alpha \to 0$ and $\alpha \to \infty$, corresponding to modularity. For $\alpha \in (0, 1)$, the Rényi entropy is neither supermodular nor submodular, as verified via numerical examples demonstrating explicit violations of both inequalities.

The proof employs Lorenz curve geometry to define glb and lub, leveraging Jensen's inequality under convex transformations of partial sums. The modular cases correspond to the cardinality of support sets (for $$H_\alpha(p \wedge q) \leq H_\alpha(p) + H_\alpha(q)$$0) and largest probability masses (for $$H_\alpha(p \wedge q) \leq H_\alpha(p) + H_\alpha(q)$$1).

Importantly, supermodularity does not extend beyond pairs: for three or more PMFs, the inequality fails.

Information-Theoretic Distance and Econometric Applications

The authors propose an information-theoretic distance between PMFs,

$$H_\alpha(p \wedge q) \leq H_\alpha(p) + H_\alpha(q)$$2

which, for $$H_\alpha(p \wedge q) \leq H_\alpha(p) + H_\alpha(q)$$3, recovers the Shannon-entropy-based distance and, for uniform distributions, matches the Theil index. This distance family is parametrized by $$H_\alpha(p \wedge q) \leq H_\alpha(p) + H_\alpha(q)$$4, offering tunable sensitivity to deviations from uniformity. Larger values of $$H_\alpha(p \wedge q) \leq H_\alpha(p) + H_\alpha(q)$$5 emphasize dominant components, providing a nuanced measure for inequality analysis. The extension to applications in econometrics, source coding, quantum entanglement, and statistical modeling underscores the practical utility of these theoretical developments.

Implications and Future Directions

The results provide structural insights into the interplay between entropy measures and majorization, extending classical information inequalities to a broader entropic landscape. The proofs, grounded in coupling theory and Lorenz curve geometry, yield transparent routes to entropy inequalities on ordered spaces.

Potential future directions include empirical study and statistical analysis of Rényi-parametrized entropy distances, algorithmic developments for PMF aggregation and coupling optimization, and further exploration in quantum information theory (e.g., catalytic majorization and entanglement conversion). Additionally, the dependence of inequalities on entropy order highlights the value of selecting appropriate parameters for application-specific needs in measurement and inference.

Conclusion

The paper rigorously establishes subadditivity and supermodularity properties of Rényi entropy across the majorization lattice. The coupling relation between comonotone and independent structures is leveraged to derive strong entropy inequalities. The introduction of Rényi-parametrized distance metrics opens avenues for econometric analysis and practical measurement of inequality, emphasizing the importance of entropy order selection within diverse applications. The results consolidate and extend the structural calculus of entropic functions on ordered spaces, laying groundwork for theoretical and applied developments in information theory and related domains.