A new group of transformations related to the Kullback-Leibler and Rényi divergences and universal classes of monotone measures of statistical complexity (2512.11594v1)

Abstract: In this work we introduce a family of transformations, named \textit{divergence transformations}, interpolating between any pair of probability density functions sharing the same support. We prove the remarkable property that the whole family of Kullback-Leibler and Rényi divergences evolves in a monotone way with respect to the transformation parameter. Moreover, fixing the reference density, we show that the divergence transformations enjoy a group structure and can be derived through the algebraic conjugation of the recently introduced differential-escort transformations and their relative counterparts. This algebraic structure allows us to deform any density function in such a way its divergence with respect a fixed reference density might also increase as much as possible. We also establish the monotonicity of composed measures involving the proper Kullback-Leibler and Rényi divergences as well as other recently introduced relative measures of moment and Fisher types. As applications, an approximation scheme of general density functions by simple functions is provided. In addition, we give a number of analytical and numerical examples of interest in both regimes of increasing and decreasing divergence.

• The paper introduces a new group of divergence transformations that interpolate between densities, ensuring controlled monotonicity of KL and Rényi divergences.
• It establishes a non-commutative group structure via algebraic properties, unifying differential-escort and relative transformation methods.
• Analytical and numerical examples illustrate the transformations' utility in density approximation, complexity analysis, and anomaly detection.

A New Group Structure for Divergence Transformations and Monotone Measures of Statistical Complexity

Introduction and Theoretical Motivation

This work introduces and rigorously analyzes a new group of transformations—termed divergence transformations—that interpolate between probability densities while controlling the Kullback-Leibler (KL) and Rényi divergences in a monotonic manner (2512.11594). These transformations provide a systematic framework for deforming a density $f$ with respect to a reference density $h$, such that information-theoretic divergence measures evolve monotonically with the transformation parameter. This approach offers an overview and algebraic unification of several transformation concepts previously developed in mathematical physics and information theory, including differential-escort and relative differential-escort transformations.

Such transformations have broad implications: they naturally generate universal, monotone classes of statistical complexity measures, and they establish a group-theoretical foundation for the construction of deformations that are maximally discriminative with respect to divergence-based metrics. The work gives a full algebraic and analytic characterization of these transformations, along with their implications for monotonicity, universality, and group structure in the context of statistical complexity.

Construction of Divergence Transformations

The framework considers pairs of strictly positive PDFs $f$ and $h$ on a common support $\Omega$ and introduces the family

$$\mathfrak{A}_{\alpha}^{(h)}[f](y) = K_{\alpha}[h \| f] \left(\frac{f(x(y))}{h(x(y))}\right)^{\alpha} h(y)$$

where the change of variable $y(x)$ is constructed via normalization of mixed powers of $f$ and $h$. The constant $K_{\alpha}[h \| f]$ ensures total probability preservation, defined as

$$K_{\alpha}[h \| f] = \int_{\Omega} h(x)^{\alpha} f(x)^{1-\alpha} dx.$$

The transformation parameter $\alpha$ interpolates between the reference ($\alpha = 0$, yielding $h$) and the input density ($\alpha = 1$, yielding $f$), with monotonic divergence behavior throughout.

A key algebraic result is that this transformation possesses a non-commutative group structure with (for $\alpha \ne 0$)

$$\mathfrak{A}_{\beta}^{(h)} \circ \mathfrak{A}_{\alpha}^{(h)} = \mathfrak{A}_{\beta\alpha}^{(h)}, \qquad (\mathfrak{A}_{\alpha}^{(h)})^{-1} = \mathfrak{A}_{1/\alpha}^{(h)}.$$

This group property is inherited from the algebraic structure of differential-escort and relative transformations and positions the divergence transformations as universal, canonical interpolators for information-theoretic metrics.

Monotonicity of Divergences and Universal Complexity Measures

An essential analytic result established in the paper is the monotonicity of both KL and Rényi divergences under these transformations: $$\alpha \mapsto D_{\xi}[\mathfrak{A}_{\alpha}^{(h)}[f] \| h]$$ is monotonically increasing for $|\alpha|$ away from 0 ($\xi \geq 1$), with $D_{\xi}[h \| h] = 0$ as the minimal point. This monotonic behavior extends not only to divergences but also to composed complexity measures, including the extended LMC-Rényi complexity—a form involving ratios of Rényi divergence powers—relative Fisher divergence measures, and cumulative moment-based measures. For all these, the paper supplies explicit expressions, analytic monotonicity, and a full characterization of equality cases.

Specifically, for the LMC-Rényi complexity, the transformation satisfies: $$C_{\lambda,\beta}^{(D)}[\mathfrak{A}_{\alpha}^{(h)}[f] \| h] = \left( C_{\lambda_\alpha, \beta_\alpha}^{(D)}[f \| h] \right)^{\alpha}$$ where $C_{\lambda,\beta}^{(D)}$ denotes the relative LMC-Rényi complexity built from Rényi divergences of order $\lambda$ and $\beta$.

In addition, utilizing conjugations with up/down transformations, the authors construct new classes of monotone complexity measures involving relative Fisher information and cumulative moments, exhibiting parallel monotonicity properties and saturations for special density/reference cases.

Analytical and Numerical Examples

The analytic tractability of divergence transformations is demonstrated on classical families: power-law, exponential, Gaussian, Beta, and piecewise-constant densities. The paper provides explicit forms and critical parameter calculations demarcating well-defined versus singular (delta-like) limiting behaviors, thus mapping the transformation landscape in function space.

Notably, for $N$-piecewise densities, the authors describe an approximation scheme whereby repeated application of the divergence transformation can push any simple density arbitrarily close to the reference, in both analytic and numerical implementation. This underlines the utility of the transformation as a universal tool for density approximation, refinement, or separation, depending on the parameter regime.

Numerically, the paper illustrates the controlled approach and separation of densities by tuning $\alpha$, showing, for instance, the smooth convergence of a piecewise density toward a Gaussian, and, conversely, the amplification of asymmetry or tail differences between a skewed input and a reference—thereby highlighting direct algorithmic applications for change detection, anomaly recognition, and structure analysis in data.

Practical and Theoretical Implications

The introduction of a group of divergence transformations with globally monotone divergence properties yields several implications:

• Divergence Optimization: The transformations provide a canonical method to maximally increase or decrease (relative to a reference) a divergence functional, which is directly applicable in hypothesis testing, density estimation, and outlier analysis.
• Statistical Complexity Analysis: The paper establishes new universal monotone complexity measures, with direct connections to the LMC, Fisher, and moment-based complexity indices, facilitating systematic exploration of order/disorder transitions in physical and synthetic datasets.
• Density Approximation and Deformation: The analytic tractability across distribution families enables their use as density approximation or modification tools, important for statistical modeling and simulation where precise control over divergence properties is desired.
• Framework for Informational Inequalities: By connecting with classical informational inequalities and establishing group-theoretic structure, the paper provides potential for future generalizations in the theory of $f$-divergences, Fisher information, and non-extensive entropy functionals.

From a computational perspective, the explicit formulas and invertibility properties suggest efficient algorithms for reference-centered density interpolation and statistical discrimination tasks, especially for high-dimensional or nonparametric applications.

Conclusion

This work establishes a new, mathematically rigorous group of divergence transformations with explicit monotonicity properties for both Kullback-Leibler and Rényi divergences, and demonstrates how these underpin universal and monotone measures of statistical complexity. Through the unifying algebraic framework, explicit analytical formulas, and applications to common distribution families, the paper provides powerful new tools for both theoretical analysis and practical manipulation of probability densities in information theory, mathematical physics, and data analysis (2512.11594). The general framework laid out paves the way for extensions to other divergence measures, relative information functionals, and potential applications in statistical inference, signal processing, and anomaly detection.