On Higher-Order Geometric Refinements of Classical Covariance Asymptotics: An Approach via Intrinsic and Extrinsic Information Geometry

Abstract: Classical Fisher-information asymptotics describe the covariance of regular efficient estimators through the local quadratic approximation of the log-likelihood, and thus capture first-order geometry only. In curved models, including mixtures, curved exponential families, latent-variable models, and manifold-constrained parameter spaces, finite-sample behavior can deviate systematically from these predictions. We develop a coordinate-invariant, curvature-aware refinement by viewing a regular parametric family as a Riemannian manifold ((Θ,g)) with Fisher--Rao metric, immersed in (L^2(μ)) through the square-root density map. Under suitable regularity and moment assumptions, we derive an (n^{-2}) correction to the leading (n^{-1}I(θ)^{-1}) covariance term for score-root, first-order efficient estimators. The correction is governed by a tensor (P_{ij}) that decomposes canonically into three parts, an intrinsic Ricci-type contraction of the Fisher--Rao curvature tensor, an extrinsic Gram-type contraction of the second fundamental form, and a Hellinger discrepancy tensor encoding higher-order probabilistic information not determined by immersion geometry alone. The extrinsic term is positive semidefinite, the full correction is invariant under smooth reparameterization, and it vanishes identically for full exponential families. We then extend the picture to singular models, where Fisher information degenerates. Using resolution of singularities under an additive normal crossing assumption, we describe the resolved metric, the role of the real log canonical threshold in learning rates and posterior mean-squared error, and a curvature-based covariance expansion on the resolved space that recovers the regular theory as a special case. This framework also suggests geometric diagnostics of weak identifiability and curvature-aware principles for regularization and optimization.

• The paper introduces a refined, coordinate-invariant second-order covariance expansion using intrinsic and extrinsic information geometry.
• It leverages the Hellinger immersion and curvature tensors to quantify deviations from standard Fisher information asymptotics.
• It extends the geometric approach to singular models via resolution of singularities, offering robust insights for finite-sample inference.

Higher-Order Geometric Refinements of Covariance Asymptotics via Information Geometry

Introduction and Motivation

The paper "On Higher-Order Geometric Refinements of Classical Covariance Asymptotics: An Approach via Intrinsic and Extrinsic Information Geometry" (2604.12725) develops a refined coordinate-invariant second-order asymptotic theory of estimator covariance for regular and singular statistical models. Classical Fisher-information asymptotics, based on local quadratic approximation of the log-likelihood, offer an $n^{-1}I(\theta)^{-1}$ covariance expansion, reflecting only first-order geometry. However, statistical models with curvature—such as finite mixtures, latent-variable models, and manifold-constrained families—demonstrate systematic second-order deviations from these predictions.

This work leverages information geometry: it models the statistical manifold $(\Theta, g)$ (with $g$ the Fisher--Rao metric) as an immersed Riemannian manifold in Hilbert space via the square-root density (Hellinger) map. The resulting geometric formalism interrogates both intrinsic curvature (via the Riemann tensor of the Fisher--Rao metric) and extrinsic curvature (via the second fundamental form of the square-root density immersion), which are shown to govern higher-order corrections to asymptotic covariance.

Geometric Framework: Statistical Manifold and Hellinger Immersion

A parametric family $\{p_\theta : \theta \in \Theta\} \subset L^2(\mu)$ is realized via the Hellinger map $\psi_\theta = \sqrt{p_\theta}$, yielding an immersed statistical manifold in the unit sphere of $L^2(\mu)$. The tangent space at $\psi_\theta$ is spanned by $$\partial_i\psi_\theta = \frac{1}{2}\sqrt{p_\theta}\partial_i\log p_\theta$$, and the induced Riemannian metric is the Fisher information: $$g_{ij}(\theta) = \mathbb{E}_\theta[\partial_i \log p_\theta(X)\partial_j \log p_\theta(X)] = I_{ij}(\theta)$$. This construction is invariant under smooth reparameterization and forms the basis for information-theoretic notions of distinguishability.

Curvature Structures: Intrinsic and Extrinsic Geometry

Intrinsic curvature of $(\Theta, g)$ is codified by the Levi--Civita connection and the Riemann curvature tensor $(\Theta, g)$0. Nonzero curvature signals non-flattenability; local quadratic Fisher geometry cannot globally encapsulate the statistical model.

Extrinsic curvature arises from the second fundamental form $(\Theta, g)$1 under the Hellinger immersion into $(\Theta, g)$2. The Gauss equation relates intrinsic curvature to extrinsic bending: for $(\Theta, g)$3 immersed in a flat space, $(\Theta, g)$4 is precisely determined by quadratic contractions of $(\Theta, g)$5. The scalar extrinsic curvature $(\Theta, g)$6 is a natural positive-definite measure quantifying deviation from local flatness.

In one-dimensional models, Efron's statistical curvature $(\Theta, g)$7 quantifies second-order loss in Fisher information, establishing a link between extrinsic geometry and higher-order efficiency.

Higher-Order Likelihood Expansions and Tensorial Structure

The paper leverages Taylor expansions of the log-likelihood, introducing the score, Hessian, and third-order score tensors. Crucially, the observed Hessian is not a genuine tensor under reparameterization; coordinate invariance is achieved by the covariant Hessian. Third-order moments and exponential-connection coefficients (Amari--Chentsov tensor) capture departures from quadraticity, encoding information relevant to second-order asymptotics.

Second-Order Covariance Expansion: Canonical Geometric Decomposition

For first-order efficient estimators, the paper derives an explicit $(\Theta, g)$8 correction to covariance, organized via a canonical tensor $(\Theta, g)$9: $g$0 where:

• $g$1: Ricci-type intrinsic contraction of the Fisher--Rao Riemann curvature tensor
• $g$2: Gram-type extrinsic contraction of the second fundamental form (always positive semidefinite)
• $g$3: Hellinger discrepancy tensor, accounting for score moments not encoded by immersion geometry

The full covariance expansion is: $g$4 Key structural results include:

• The correction vanishes in full exponential families ($g$5).
• Positive semidefiniteness of $g$6.
• In one dimension, only the extrinsic term remains, aligning with Efron's curvature.

Notably, $g$7 quantifies the second-order inefficiency induced by curvature, shining theoretical light on weak identifiability and practical guidance for model selection and regularization.

Extension to Singular Models via Resolution of Singularities

The paper generalizes the geometric covariance expansion to singular statistical models, where the Fisher information degenerates and tangent space acquires a cone structure rather than a linear space. By resolving singularities (Hironaka theory), one expresses the Kullback--Leibler divergence in normal crossing form on a resolved manifold. The real log canonical threshold (RLCT), a key algebraic invariant, governs the rate of posterior contraction and estimator learning rates.

The geometric machinery—metric, connection, and curvature—is pushed forward from the resolved space to the original parameter manifold. The asymptotic covariance expansion adapts accordingly: $g$8 where $g$9 and $\{p_\theta : \theta \in \Theta\} \subset L^2(\mu)$0 emerge from the singularity's algebraic structure, and $\{p_\theta : \theta \in \Theta\} \subset L^2(\mu)$1, $\{p_\theta : \theta \in \Theta\} \subset L^2(\mu)$2, $\{p_\theta : \theta \in \Theta\} \subset L^2(\mu)$3 are effective metrics and curvature tensors on the resolved space.

This singular counterpart recovers the classical expansion as a special case and encompasses models where traditional Fisher theory breaks down.

Practical and Theoretical Implications

From a practical standpoint, these geometric corrections are significant for finite-sample analysis, weak identifiability diagnostics, and regularization in probabilistic learning systems. The curvature-aware expansion indicates that overparameterized models or those with strong geometric constraints necessitate second-order efficiency diagnostics beyond Fisher information.

Potential applications include:

• Curvature-based regularization: Penalizing Ricci contraction or extrinsic curvature to favor models that are second-order stable.
• Optimization approaches: Refining natural gradient methods with curvature correction, if computationally tractable.
• Diagnostic tools: Monitoring geometric tensors ($\{p_\theta : \theta \in \Theta\} \subset L^2(\mu)$4, $\{p_\theta : \theta \in \Theta\} \subset L^2(\mu)$5, $\{p_\theta : \theta \in \Theta\} \subset L^2(\mu)$6) to identify weak identifiability and optimize solution selection.
• Deep learning settings: Applying curvature-aware heuristics for sharpness and flatness in minima selection, informed by second-order geometry.

Current obstacles include computational tractability in high dimensions; only specific contractions of curvature tensors may be efficiently estimated.

Conclusion

This work provides a rigorous, coordinate-invariant geometric refinement of classical covariance asymptotics, with explicit second-order corrections controlled by intrinsic and extrinsic curvature. The formalism transcends standard Fisher information analysis, identifying the exact mechanism by which model curvature impacts estimator efficiency at finite sample sizes. The singular case is accommodated algebraically and geometrically, yielding a unified framework that holds practical value for modern statistical inference and learning theory. Future directions include improved computational approximations for high-dimensional models, extensions to non-score-root estimators, and actionably incorporating geometric diagnostics and regularization in large-scale probabilistic systems.