Eguchi's Theory: Geometric Foundations

• Eguchi’s Theory is a framework that defines geometric structures, such as Riemannian metrics and dual affine connections, from divergence functions on statistical manifolds.
• It generalizes classical statistical concepts like the Fisher information metric and Cramér–Rao bound by incorporating divergences such as relative α-entropy and Rényi divergence.
• The theory unifies projection theorems and robust estimation methods through the use of escort distributions and duality, bridging classical results with modern information geometry.

Eguchi’s theory provides a systematic methodology for associating divergence functions defined on statistical manifolds with Riemannian metrics and dualistic geometric structures. These geometric entities serve as the foundation for generalized versions of classical results in information theory and estimation, such as the Fisher information metric and the Cramér–Rao bound. The theory forms a conceptual bridge between various divergence-based estimation procedures and the differential geometry of statistical models, allowing unified treatment of projection theorems, dual connections, and robustness analyses within a single framework.

1. The Core of Eguchi’s Theory: Divergence Functions and Geometric Structures

Eguchi’s theory posits that any sufficiently smooth divergence function $D(p,q)$, defined on a statistical manifold $S$ (a parameterized family of probability densities), canonically induces geometric structures on $S$:

• Riemannian Metric: Defined by the negative Hessian of $D$ at coincident arguments:

$$g_{ij}^{(D)}(\theta) = -\left.\frac{\partial^2}{\partial\theta_i \partial\theta'_j} D(p_\theta, p_{\theta'})\right|_{\theta'=\theta}.$$

• Dual Affine Connections: Defined via third derivatives of the divergence, with Christoffel symbols, e.g.:

$$\Gamma_{ij,k}^{(D)}(\theta) = -\left.\frac{\partial}{\partial\theta_k} \frac{\partial^2}{\partial\theta_i\partial\theta'_j} D(p_\theta, p_{\theta'})\right|_{\theta' = \theta};$$

the dual connection is defined by interchanging arguments.

For example, when $D$ is the Kullback-Leibler (KL) divergence, the resulting metric is the Fisher information matrix:

$$g_{ij}^{(\mathrm{KL})}(\theta) = \mathbb{E}_\theta\left[ \partial_i \log p_\theta(X) \, \partial_j \log p_\theta(X) \right].$$

This construction remains valid for more general divergences, such as relative $\alpha$-entropy and Rényi divergence, with the induced metric and connections providing deformations of the Fisher metric and the (e-, m-) dual connections (Karthik et al., 2017, Kumar et al., 2020, Mishra et al., 2021, Dhadumia et al., 28 Jul 2025).

2. Divergences, Escort Distributions, and the $\alpha$-Information Metric

When generalized Csiszár $f$-divergences are used (notably, relative $\alpha$-entropy), Eguchi’s method induces geometries on the escort family $S^{(\alpha)}$, where each measure is transformed:

$$p^{(\alpha)}(x) = \frac{ [p(x)]^\alpha }{ \sum_y [p(y)]^\alpha }.$$

The Riemannian metric induced by such divergences takes the form:

$$g_{ij}^{(\alpha)}(\theta) = \mathrm{Cov}_{\theta^{(\alpha)}}\left[ \partial_i \log p_\theta(X), \partial_j \log p_\theta(X) \right],$$

or equivalently:

$$g_{ij}^{(\alpha)}(\theta) = \frac{1}{\alpha^2} \mathrm{Cov}_{\theta^{(\alpha)}}\left[ \partial_i \log p_\theta^{(\alpha)}(X), \partial_j \log p_\theta^{(\alpha)}(X) \right].$$

In the case of Rényi divergence of order $\alpha$, the metric is a scalar multiple of the Fisher information:

$$g_{ij}^{(D_\alpha)}(\theta) = \alpha\, g_{ij}^{(I)}(\theta).$$

Thus, the Fisher information metric is generalized to the $\alpha$-information metric on the escort manifold, creating a basis for subsequent generalization of statistical estimation theory (Karthik et al., 2017, Kumar et al., 2020).

3. Duality, Projections, and Pythagorean Theorems

One of the central results established via Eguchi’s theory is the equivalence of projection theorems ("Pythagorean" theorems) for different divergences under the escort correspondence. Given a convex set $C$ in the probability simplex and a divergence $D$, the projection of $Q$ onto $C$ (minimizing $D(P, Q)$ over $P \in C$) admits a Pythagorean relationship. For relative $\alpha$-entropy, this becomes:

$$\mathcal{I}_\alpha(P, Q) \geq \mathcal{I}_\alpha(P, P_*) + \mathcal{I}_\alpha(P_*, Q), \qquad \forall P \in C.$$

Due to the identity

$$\mathcal{I}_\alpha(P, Q) = D_{1/\alpha}(P^{(\alpha)} \| Q^{(\alpha)}),$$

this projection theorem is equivalent to its counterpart for Rényi divergence on the escort space, after properly rescaling convexity by $\alpha$. Therefore, geometric and approximation properties for projections in one divergence framework directly translate to another (Karthik et al., 2017).

4. Application to Generalized Cramér–Rao Inequalities

The induced metric via Eguchi’s construction enables generalization of the Cramér–Rao lower bound (CRLB). Whereas the classical CRLB uses the Fisher information, the generalized bound is based on the $\alpha$-information metric or a similar metric arising from the chosen divergence:

$$\mathrm{Var}_{p^{(\alpha)}}[\hat{\theta}^{(\alpha)}(X)] \geq [G^{(\alpha)}(\theta)]^{-1}.$$

In the robust context, such as estimation under contamination, escort distributions are used to down-weight the influence of outliers. For robust divergences, such as the Basu–Harris–Hjort–Jones (BHHJ) divergence, the induced metric and the corresponding generalized CRLB naturally account for robustness via the parameter $\alpha$:

$$g_{ij}^{(\alpha)}(\theta) = \mathbb{E}_\theta\left[ p_\theta(X)^{\alpha} \partial_i \log p_\theta(X) \partial_j \log p_\theta(X) \right].$$

The generalized CRLB, in this setting, bounds the covariance of unbiased estimators under the escort distribution, and reduces to the classical CRLB as $\alpha \to 0$ (Dhadumia et al., 28 Jul 2025, Kumar et al., 2020, Mishra et al., 2021).

5. Dual Affine Connections and the Role of Divergence Functions

Eguchi’s theory extends beyond metrics to produce dual pairs of affine connections. For a divergence $D$, one obtains connections $\nabla^{(D)}$ and $\nabla^{(D*)}$ whose Christoffel symbols are given by third derivatives:

$$\Gamma_{ij,k}^{(D)}(\theta) = -\left. \frac{\partial^3}{\partial \theta_k \partial \theta_i \partial \theta'_j} D(p_\theta, p_{\theta'}) \right|_{\theta' = \theta}.$$

These connections generalize the exponential (e-) and mixture (m-) connections that underlie Amari-Nagaoka’s dually flat geometry for exponential and mixture models. When the divergence is KL, the resulting structure is the classical one; for general $f$-divergences (e.g., relative $\alpha$-entropy), the resulting structure is a "deformation" adapted to the escort family (Kumar et al., 2020, Mishra et al., 2021).

6. Unification and Broader Applications

Eguchi’s theory provides a unified geometric perspective on statistical inference, bridging estimation theory, robust statistics, and information geometry:

• Projection theorems (Pythagorean properties) and information-geometric inequalities are equivalently expressed across a broad family of divergences whenever connected via escort or scaling correspondences (Karthik et al., 2017).
• Duality relations for estimators and efficient estimation procedures for both classical and escort models follow directly from the induced geometric structure of the manifold (Kumar et al., 2020).
• Bayesian analogues (e.g., Bayesian $\alpha$-CRLB) are developed by modifying the divergence and prior structure, with the corresponding metric and inequalities following the same principle (Mishra et al., 2021).
• The framework generalized by robust divergences (e.g., BHHJ) accommodates the influence of outliers within the metric and the variance bound, providing theoretical underpinning for robust estimation (Dhadumia et al., 28 Jul 2025).

This entire synthesis clarifies that the choice of divergence function determines all subsequent geometric, statistical, and projection properties of the model manifold. Eguchi's theory thus underpins much of modern information geometry, enabling systematic derivation and comparison across a spectrum of statistical estimation problems.