Fisher–Rao Manifold & Information Geometry

• Fisher–Rao manifold is a geometric framework for probability distributions defined by the Fisher information metric, capturing inherent statistical distinguishability.
• It leverages orthogonal decomposition to derive a finite-dimensional Covariate Fisher Information Matrix, enabling tractable computations in high-dimensional settings.
• The approach connects KL divergence curvature with natural gradients, offering practical insights for model selection and dimensionality estimation in statistical inference.

The Fisher–Rao manifold is the fundamental Riemannian structure on spaces of probability distributions, induced by the Fisher information metric. In both finite- and infinite-dimensional settings, it organizes families of probability densities into a geometric object whose local and global properties encode statistical distinguishability, intrinsic regularity, and constraints on estimation. This geometric viewpoint underlies a vast range of results in information geometry, statistical inference, machine learning, and mathematical physics.

1. Metric Structure of the Infinite-Dimensional Fisher–Rao Manifold

Let $$M=\{f:\mathbb{R}^n\to\mathbb{R}_+~|~f~\text{smooth},~\int f=1\}$$ denote the smooth statistical manifold of all probability densities on $\mathbb{R}^n$. The tangent space at each $f\in M$ is

$$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$

Every $h\in T_f M$ uniquely corresponds to a mean-zero "score function" $s(x)=h(x)/f(x)$, yielding a linear isomorphism between tangent space elements and $L^2(f)$ mean-zero functions.

The Fisher–Rao metric is the unique (up to scale) diffeomorphism-invariant Riemannian metric on $M$:

$$g_f(h_1,h_2) = \int_{\mathbb{R}^n} \frac{h_1(x)h_2(x)}{f(x)} dx = \mathbb{E}_f[s_1(X)s_2(X)].$$

This operator is a (degenerate) integral operator in infinite dimensions, coupling all points in $\mathbb{R}^n$, with no closed-form inverse in general. This non-invertibility produces an "intractability barrier" in explicit computations of geodesics, curvature, or natural gradients on $\mathbb{R}^n$0 (Cheng et al., 25 Dec 2025).

2. Covariate Fisher Information Matrix via Orthogonal Decomposition

Finite observables $\mathbb{R}^n$1 induce a finite-dimensional subspace of $\mathbb{R}^n$2:

$\mathbb{R}^n$3

$\mathbb{R}^n$4 and its $\mathbb{R}^n$5-orthogonal complement $\mathbb{R}^n$6 provide a Hilbert space splitting:

$\mathbb{R}^n$7

ensuring any $\mathbb{R}^n$8 is decomposed as $\mathbb{R}^n$9, with $f\in M$0.

Restricting $f\in M$1 to $f\in M$2 yields the finite-dimensional Covariate Fisher Information Matrix (cFIM), $f\in M$3:

$f\in M$4

This matrix is positive-definite and invertible under generic linear independence assumptions of the scores.

This finite-dimensional restriction enables:

• tractable preconditioning of gradients,
• explicit computation of the “explainable” Fisher information,
• algorithmic access to local geometric invariants (Cheng et al., 25 Dec 2025).

3. Geometric and Statistical Invariants: G-Entropy, Curvature, and Efficiency

The trace of the covariate Fisher information matrix,

$f\in M$5

is the G-entropy. The Trace Theorem establishes $f\in M$6, characterizing the total explainable statistical information as a gradient-based geometric invariant. Further, this trace equals the total curvature of the Kullback–Leibler (KL) divergence along each covariate direction.

The metric $f\in M$7 is intrinsically linked to the curvature (i.e., second derivative) of the KL divergence:

$f\in M$8

for smooth paths $f\in M$9 with tangent $$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$0. Restricting $$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$1 to $$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$2 identifies $$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$3 as the local Hessian of KL in observable directions, thus justifying its use in sensitivity analysis and statistical curvature (Cheng et al., 25 Dec 2025).

Under geometric alignment in semi-parametric inference—when the efficient score aligns with the span of covariate scores—$$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$4 coincides with the efficient Fisher information, yielding the optimal Cramér–Rao bound for regular asymptotically linear estimators:

$$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$5

and the optimal influence function is $$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$6.

4. Information Capture Ratio and the Manifold Hypothesis

The orthogonal decomposition gives rise to the information capture ratio for arbitrary perturbations $$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$7:

$$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$8

quantifying the proportion of Fisher information explained by the observable covariate directions.

For high-dimensional distributions, the manifold hypothesis posits mass is concentrated near a $$T_f M = \{h:\mathbb{R}^n\to\mathbb{R}~|~\int h(x)\,dx = 0\}.$$9 dimensional submanifold $h\in T_f M$0. In the information-geometric framework, this translates into numerical rank-deficiency of $h\in T_f M$1: the tangent space of $h\in T_f M$2 is nested within $h\in T_f M$3, and generic perturbations outside $h\in T_f M$4 have vanishing Fisher mass. Empirically, the eigenvalue spectrum $h\in T_f M$5 of an estimated $h\in T_f M$6 from data reveals “intrinsic dimension” if there is sharp decay: the effective rank $h\in T_f M$7 is evidenced by $h\in T_f M$8 (Cheng et al., 25 Dec 2025).

This gives a principled, spectral-theoretic, and testable method for dimension estimation, connecting the abstract theory to practical machine learning and manifold learning tasks.

5. Computational Implications and Statistical Inference

The positive-definite $h\in T_f M$9 enables inversion and tractable computation of natural gradients in the $s(x)=h(x)/f(x)$0-direction, circumventing intractable infinite-dimensional inversion. This supplies a practical tool for statistical learning and inference in infinite-dimensional, nonparametric settings, while ensuring that statistical risk and estimator variance are locally governed by $s(x)=h(x)/f(x)$1 and its spectrum (Cheng et al., 25 Dec 2025).

Furthermore, for semi-parametric models, the developed framework directly identifies the efficient Fisher information and optimal estimators in terms of the cFIM, providing explicit geometric justification for both global and local efficiency bounds.

6. Broader Impact and Connections

The construction in (Cheng et al., 25 Dec 2025) bridges pure information geometry with practical model selection, explainability, and learning theory. By resolving the infinite-dimensional intractability of the Fisher–Rao metric via orthogonal decomposition and the cFIM, it enables spectral diagnostics for model dimensionality, interpretable regularization (via G-entropy), and efficient statistical inference.

This approach also grounds the manifold hypothesis in rigorous geometric criteria, moving from heuristic assumptions to robust and testable rank-deficiency in eigenvalue spectra.

The methodology provides a template for extending parametric and semi-parametric statistical theory—based on the Fisher–Rao manifold, efficient Fisher information, and KL curvature—to the genuinely nonparametric setting via computable, intrinsic finite-dimensional surrogates.

---

References:

• (Cheng et al., 25 Dec 2025) An approach to Fisher-Rao metric for infinite dimensional non-parametric information geometry