Mean Information Dimension

• Mean information dimension is a process-level measure derived from the rate-distortion function that quantifies the bits per iterate needed at small distortions.
• It connects various dimensional theories—like metric mean and Hausdorff dimensions—by formalizing the small-distortion behavior in both dynamical and stochastic settings.
• The concept is extended to stationary processes with precise formulations for Gaussian cases, exposing differences from traditional entropy rates and underscoring the role of spectral properties.

Searching arXiv for recent and foundational papers on mean information dimension, rate-distortion dimension, and metric mean dimension. Mean information dimension is the dynamical or process-level analogue of Rényi’s information dimension, defined by the small-distortion slope of a rate-distortion function or, equivalently in the stationary process setting, by the growth rate of the entropy rate of a uniformly quantized process. In the mean-dimension literature it is usually formalized as the rate-distortion dimension of a measure-preserving dynamical system, while in stochastic-process theory it appears as the information dimension rate; the two viewpoints are linked explicitly in the high-resolution regime (Lindenstrauss et al., 2019, Geiger et al., 2017). In symbolic dynamics and mean dimension theory, mean information dimension functions as an information-theoretic counterpart to metric mean dimension and mean Hausdorff dimension, and in several important classes of systems it admits exact entropy formulas (Shinoda et al., 2019).

1. Formal definitions and basic quantities

Let $(\mathcal{X},T)$ be a compact metrizable dynamical system, let $d$ be a compatible metric, and define the Bowen metrics

$d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$

The scale-$\varepsilon$ dynamical covering complexity is

$$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$

where $\#(\mathcal{X},d_N,\varepsilon)$ is the $\varepsilon$-covering number. The upper and lower metric mean dimensions are then

$$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$

When the two coincide, the common value is denoted $\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$ (Lindenstrauss et al., 2019).

For a $T$-invariant Borel probability measure $d$0, the rate-distortion function at distortion level $d$1 is

$d$2

where $d$3, $d$4 takes values in $d$5, and

$d$6

The associated upper and lower rate-distortion dimensions are

$d$7

When they coincide, the common value is written $d$8 (Lindenstrauss et al., 2019).

In the process-theoretic literature, the same small-distortion slope is called the information dimension rate or mean information dimension. For a stationary process $d$9, it is given by

$d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$0

when the limit exists, where $d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$1 and $d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$2 denotes entropy rate (Geiger et al., 2017).

Quantity	Definition type	Interpretation
$d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$3	Covering growth under $d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$4	Dimension per iterate
$d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$5	Small-distortion slope of $d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$6	Mean information dimension
$d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$7	Shrinking-partition entropy growth	Dynamical Rényi information dimension

In the cited works, logarithms are taken in base $d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$8 (Lindenstrauss et al., 2019, Shinoda et al., 2019).

2. Relation to mean dimension and geometric dimension theories

Mean information dimension is part of a triad consisting of metric mean dimension, mean Hausdorff dimension, and rate-distortion dimension. Mean dimension itself is a topological invariant defined through width dimension:

$d_N(x,y)=\max_{0\le n<N} d(T^n x,T^n y).$9

and is independent of the compatible metric (Lindenstrauss et al., 2019).

The basic comparison inequalities place rate-distortion dimension below metric mean dimension:

$\varepsilon$0

for all invariant measures $\varepsilon$1 (Lindenstrauss et al., 2019). When the metric has tame growth of covering numbers, meaning that for every $\varepsilon$2,

$\varepsilon$3

one also has the reverse-type variational bound

$\varepsilon$4

where $\varepsilon$5 is mean Hausdorff dimension (Lindenstrauss et al., 2019). This places mean information dimension at the interface between information theory and geometric measure theory.

A central result is the double variational principle: if $\varepsilon$6 has the marker property, then

$\varepsilon$7

and the minimum over metrics is attained (Lindenstrauss et al., 2019). Thus mean dimension can be recovered as a minimax of mean information dimension over compatible metrics and invariant measures.

The marker property is the condition that for every $\varepsilon$8 there exists an open set $\varepsilon$9 such that

$$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$0

It implies freeness and is satisfied by free minimal systems and their extensions (Lindenstrauss et al., 2019).

3. Dynamical Rényi information dimension

A distinct but closely related formulation replaces mutual information by measure-theoretic entropy of small-diameter partitions. For a finite Borel partition $$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$1, denote by $$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$2 its Kolmogorov–Sinai entropy rate. The upper and lower mean Rényi information dimensions are

$$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$3

$$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$4

These were introduced as a dynamical version of Rényi information dimension through partition entropies (Yang et al., 2022, Gutman et al., 2020).

A key advantage of the partition formulation is that it yields a variational principle for metric mean dimension without tame growth assumptions:

$$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$5

and similarly for the lower metric mean dimension with $$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$6 (Gutman et al., 2020). In this sense, metric mean dimension is the supremum over invariant measures of a mean information-dimension rate defined via shrinking partitions.

The relation between the partition-based and rate-distortion-based formulations is controlled by inequalities such as

$$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$7

which shows that the partition-based mean Rényi information dimension dominates the $$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$8 rate-distortion dimension (Yang et al., 2022). Under the marker property, a further double variational principle holds:

$$S(\mathcal{X},T,d,\varepsilon)=\lim_{N\to\infty}\frac{\log \#(\mathcal{X},d_N,\varepsilon)}{N},$$9

in the sense that the upper and lower versions match in the min–sup identity (Yang et al., 2022).

For metrics $\#(\mathcal{X},d_N,\varepsilon)$0 that realize mean dimension, namely

$\#(\mathcal{X},d_N,\varepsilon)$1

the order of $\#(\mathcal{X},d_N,\varepsilon)$2 and $\#(\mathcal{X},d_N,\varepsilon)$3 can be interchanged under the marker property (Yang et al., 2022). This identifies a regime in which topological and measure-theoretic formulations of mean information dimension coincide exactly.

4. Symbolic dynamics and exact entropy formulas

The symbolic setting provides explicit formulas that closely parallel Furstenberg’s one-sided entropy-dimension theorem. Let $\#(\mathcal{X},d_N,\varepsilon)$4 be a finite alphabet and $\#(\mathcal{X},d_N,\varepsilon)$5 a $\#(\mathcal{X},d_N,\varepsilon)$6-subshift with commuting shifts $\#(\mathcal{X},d_N,\varepsilon)$7. For $\#(\mathcal{X},d_N,\varepsilon)$8, the standard symbolic metric is

$\#(\mathcal{X},d_N,\varepsilon)$9

Under the $\varepsilon$0-subaction $\varepsilon$1, Shinoda and Tsukamoto proved

$\varepsilon$2

and for every measure $\varepsilon$3 invariant under both $\varepsilon$4 and $\varepsilon$5,

$\varepsilon$6

Hence, for a measure of maximal entropy, mean information dimension, mean Hausdorff dimension, and metric mean dimension all coincide (Shinoda et al., 2019).

The same paper gives directional formulas. For a nonzero vector $\varepsilon$7 and $\varepsilon$8, one has for the $\varepsilon$9-based symbolic metric

$$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$0

$$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$1

while for the $$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$2-radius metric $$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$3 the factor $$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$4 is replaced by $$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$5 (Shinoda et al., 2019). The paper interprets these prefactors as capturing the density of orbit traces in the plane in the norm governing the metric.

For the full $$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$6 shift $$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$7, $$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$8, so

$$\overline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\limsup_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}, \qquad \underline{\mathrm{mdim}}_{\mathrm{M}}(\mathcal{X},T,d) =\liminf_{\varepsilon\to 0}\frac{S(\mathcal{X},T,d,\varepsilon)}{\log(1/\varepsilon)}.$$9

and for the Bernoulli product measure,

$\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$0

The coefficient $\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$1 reflects two-sided time in the subaction together with the geometry of the symbolic metric; Furstenberg’s original one-sided result does not have this factor (Shinoda et al., 2019).

These formulas make the phrase “mean information dimension” literal in symbolic dynamics: the number of bits per iterate needed to describe typical orbit segments at distortion $\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$2 scales as an entropy divided by $\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$3.

5. Stochastic-process formulation and spectral characterizations

For stationary stochastic processes, mean information dimension is defined through quantized entropy rates. For an $\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$4-variate process $\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$5,

$\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$6

when the limits exist; equivalently, for stationary processes,

$\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$7

Geiger and Koch proved that this equals the rate-distortion dimension:

$\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$8

with

$\mathrm{mdim}_{\mathrm{M}}(\mathcal{X},T,d)$9

and, for stationary sources,

$T$0

Thus the information dimension rate is exactly the high-resolution slope of Shannon’s rate-distortion function (Geiger et al., 2017).

The same work gives a spectral characterization for stationary Gaussian processes. If $T$1 is stationary, $T$2-variate, and Gaussian with spectral distribution function $T$3, then

$T$4

More generally,

$T$5

with equality for Gaussian processes, so among all stationary processes with a given matrix-valued spectral distribution function, the Gaussian process has the largest information dimension rate (Geiger et al., 2017).

In the scalar Gaussian case, if $T$6 is the power spectral density, then

$T$7

the Lebesgue measure of the spectral support on which the density is positive. A bandlimited Gaussian process supported on $T$8 therefore has

$T$9

(Geiger et al., 2017).

Only the absolutely continuous part of the spectral distribution contributes to the information dimension rate. If the spectral distribution decomposes as

$d$00

then $d$01 depends only on $d$02; the discrete and singular parts contribute zero (Geiger et al., 2017). This sharply distinguishes mean information dimension from entropy rate, since a process may carry nontrivial structure in singular or discrete spectral components without increasing the small-distortion dimension rate.

6. Extensions, distinctions, and open directions

Several non-equivalent notions of process complexity coexist in the literature. For stationary processes, Geiger and Koch distinguish the information dimension rate $d$03 from the block-average notion $d$04 and show that, in general,

$d$05

with equality under additional mixing hypotheses, such as the condition that there exists $d$06 with $d$07 for all $d$08 (Geiger et al., 2017). A stationary Gaussian process can satisfy $d$09, so “mean information dimension” is not interchangeable with every block-entropy-based dimension notion.

In the shift setting on $d$10, Gutman and Śpiewak showed for ergodic invariant measures that the dynamical mean Rényi information dimension equals the Geiger–Koch information dimension rate (Gutman et al., 2020). A later result extended this to all invariant measures on $d$11:

$d$12

and, via the Geiger–Koch theorem, also to the $d$13 rate-distortion dimensions (Yang, 9 Oct 2025). This answers the non-ergodic question left open in the earlier work.

A common misconception is that positive entropy forces positive mean information dimension. Finite-alphabet symbolic systems provide a counterpoint: with the discrete metric, mean Rényi information dimension and rate-distortion dimensions can vanish even though the Kolmogorov–Sinai entropy is positive. At the same time, the associated rate-distortion entropies

$d$14

equal $d$15 for ergodic measures, and for all invariant measures under the $d$16-almost product property (Yang, 9 Oct 2025). This indicates that entropy and mean information dimension probe different asymptotic normalizations: fixed-resolution compression versus small-distortion dimensional scaling.

Recent work also sharpens the variational theory. For systems with the marker property and finite mean dimension, the supremum in the double variational principle can be restricted to ergodic measures, and analogous statements hold for a broader family of measure-theoretic $d$17-entropies, including partition-based, cover-based, Katok, Brin–Katok, and Pfister–Sullivan quantities (Yang, 9 Oct 2025). A plausible implication is that the ergodic structure of maximizing measures is more rigid than the original variational formulas alone suggest.

Several open problems remain. Lindenstrauss and Tsukamoto conjectured that the marker property may be unnecessary for the double variational principle (Lindenstrauss et al., 2019). Yang, Chen, and Zhou likewise isolate the problem of removing the marker-property hypothesis from the interchange of $d$18 and $d$19 in the Rényi-information-dimension formulation, and they raise the question of existence of maximal metric mean dimension measures for suitable candidates $d$20 (Yang et al., 2022). These questions concern whether mean information dimension can be made fully intrinsic, without auxiliary assumptions on markers or on specially chosen metrics.