---
title: 'Riemannian NML: Model Selection on Manifolds'
url: https://www.emergentmind.com/topics/riemannian-nml
type: topic
---

# Riemannian NML: Model Selection on Manifolds

Riemannian Normalized Maximum Likelihood (Rm-NML) is an extension of the Normalized Maximum Likelihood (NML) universal coding and model selection framework to statistical models where the data space is a Riemannian manifold. It provides a coordinate-invariant, geometrically consistent notion of stochastic complexity and regret minimization in non-Euclidean settings. Rm-NML recovers the conventional NML code-length in Euclidean spaces and enables model selection and information-theoretic analysis for data distributed on general manifolds such as hyperbolic spaces, which are of growing interest in graph and hierarchical data modeling [2508.21466].

## 1. Formal Definition and Distribution Construction

Let $(\mathcal{M}, g)$ be a $D$-dimensional Riemannian manifold with metric $g$ and induced volume element $d\operatorname{vol}(x) = \sqrt{\det g(x)}\,dx$. Consider a parametric family of model densities $p_\operatorname{vol}(x|\theta)$, defined with respect to $d\operatorname{vol}(x)$, for $x \in \mathcal{M}$, $\theta \in \Theta$. The maximum-likelihood estimator is
$$\hat\theta(x) = \arg\max_\theta p_\operatorname{vol}(x|\theta).$$

The Rm-NML distribution is then
\[
p_{\rm Rm\text{-}NML}(x) = \frac{ p_\operatorname{vol}(x|\hat\theta(x)) }{ \displaystyle \int_\mathcal{M} p_\operatorname{vol}(y|\hat\theta(y))\,d\operatorname{vol}(y) }.
\]
The associated code-length is
\[
L_{\rm Rm\text{-}NML}(x) = -\log p_\operatorname{vol}(x|\hat\theta(x)) + \log C_M,
\qquad
C_M = \int_\mathcal{M} p_\operatorname{vol}(y|\hat\theta(y))\,d\operatorname{vol}(y).
\]
This framework naturally generalizes the Shtarkov NML code to any manifold equipped with a Riemannian measure.

## 2. Coordinate Invariance and Role of the Fisher Information

The Rm-NML code-length is invariant under smooth coordinate transformations due to two geometric properties:
- The volume element $d\operatorname{vol}(x) = \sqrt{\det g(x)}\,dx$ transforms contravariantly, ensuring the scalar nature of $p_\operatorname{vol}(x|\theta) d\operatorname{vol}(x)$.
- In asymptotic normalizing constant calculations, the Fisher information metric $I(\theta)$ on $\Theta$ yields the Jeffreys prior $\sqrt{\det I(\theta)}\,d\theta$, which is itself invariant under reparameterization.

Explicitly, if $\phi$ and $\psi$ are smooth coordinate charts on $\Theta$:
\[
\sqrt{\det I_\phi(\theta)}\,d\phi(\theta) = \sqrt{\det I_\psi(\theta)}\,d\psi(\theta),
\]
reflecting coordinate invariance in the parameter space. This guarantees that Rm-NML is well-defined and interpretable regardless of coordinate representation.

## 3. Reduction to Ordinary NML in Euclidean Spaces

If $\mathcal{M} = \mathbb{R}^D$ with the standard Euclidean metric, $d\operatorname{vol}(x) = dx$ and $p_\operatorname{vol}(x|\theta) = p(x|\theta)$, so the Rm-NML reduces exactly to conventional NML:
\[
L_{\rm RmNML}(x) = -\log p(x|\hat\theta(x)) + \log \int p(y|\hat\theta(y))\,dy,
\]
matching the original Shtarkov code-length and ensuring compatibility with classical minimum description length (MDL) theory.

## 4. Asymptotic and Computational Properties

For sample size $n$ and under standard regularity conditions, the normalizing constant $C_M$ can be approximated (via saddle-point asymptotics) as:
\[
\log C_M = \frac{k}{2} \log \frac{n}{2\pi} + \log \int_\Theta \sqrt{\det I(\theta)}\,d\theta + o(1),
\]
where $k = \dim \Theta$ and $I(\theta)$ is the Fisher information:
\[
I(\theta) = \lim_{n\to\infty} \frac{1}{n} E_\theta \left[-\partial_{\theta}^2 \log p_\operatorname{vol}(x^n|\theta)\right].
\]
This result replaces Lebesgue measure with the manifold volume and incorporates explicit chart transformations, mirroring the derivation of Rissanen (1996) for stochastic complexity.

## 5. Riemannian Symmetric Spaces and Simplifications

When $\mathcal{M}$ is a Riemannian symmetric space and the density $p_\operatorname{vol}(x|\theta,\gamma)$ depends on $x, \theta \in \mathcal{M}$ only via their geodesic distance $d(x, \theta)$ plus Euclidean nuisance parameters $\gamma \in \Gamma \subseteq \mathbb{R}^m$, the Fisher information matrix exhibits a block-diagonal structure:
\[
\det I_\theta(\theta, \gamma) = C_\theta(\gamma),\quad
\det I_\gamma(\theta, \gamma) = C_\gamma(\gamma),
\]
where $C_\theta$ and $C_\gamma$ are independent of $\theta$ by manifold homogeneity. Consequently,
\[
\log C_M = \frac{D + m}{2} \log \frac{n}{2\pi} + \log \operatorname{vol}(\Theta)
+ \log \int_\Gamma \sqrt{C_\theta(\gamma)C_\gamma(\gamma)}\,d\gamma + o(1).
\]
In this setting, integration over the data manifold is replaced by the volume of the parameter manifold, simplifying computation substantially.

## 6. Explicit Hyperbolic Gaussian Case

For $D$-dimensional hyperbolic space $\mathbb{H}^D$ (curvature $-1$), the Riemannian Gaussian (R-GD) density is defined as:
\[
p_\operatorname{vol}(x|\mu, \sigma) = \frac{1}{\xi(\sigma)}\exp\left(-\frac{1}{2\sigma^2}d^2(x, \mu)\right),
\]
with the normalizing factor
\[
\xi(\sigma) = \int_{\mathbb{H}^D} \exp\left(-\frac{d^2(x,\mu)}{2\sigma^2}\right) d\operatorname{vol}(x).
\]
For data $x^n$, the MLEs are the Riemannian–Fréchet mean $\hat\mu$ and variance estimator $\hat\sigma^2 = \frac{1}{nD} \sum_{i=1}^n d^2(x_i, \hat\mu)$. The fitted log-likelihood is
\[
-\log p_\operatorname{vol}(x^n|\hat\mu, \hat\sigma) = n\log \xi(\hat\sigma) + \frac{nD}{2}.
\]
Applying the symmetric-space formula, the Rm-NML normalizing constant (Corollary 6.1 [2508.21466]) is
\[
\log C_M = \frac{D+1}{2} \log\frac{n}{2\pi} + \log \operatorname{Vol}_{\mathbb{H}^D}(R)
+ \log\int_0^\infty \left(\frac{1}{D\sigma^2} \frac{\xi'(\sigma)}{\xi(\sigma)}\right)^{D/2} B(\sigma)\,d\sigma + o(1),
\]
where $\operatorname{Vol}_{\mathbb{H}^D}(R)$ is a geodesic ball volume (for restricted $\mu$), and
\[
B(\sigma) = \sqrt{ \frac{1}{\sigma}\left[ \sigma \frac{\xi''(\sigma)}{\xi(\sigma)} 
- \sigma\left(\frac{\xi'(\sigma)}{\xi(\sigma)}\right)^2 + 3\frac{\xi'(\sigma)}{\xi(\sigma)} \right] }.
\]
The full Rm-NML code-length for hyperbolic Gaussian models is then
\[
L_{\rm RmNML}(x^n) = n\log\xi(\hat\sigma) + \frac{nD}{2} + \log C_M.
\]

## 7. Practical Computation and Applications

The remaining integrals in $C_M$—over Euclidean parameters or geodesic ball volumes—are amenable to numerical quadrature, Monte Carlo, or, in the Euclidean limit, Fourier methods. Optimization for the Fréchet mean on $\mathbb{H}^D$ is addressed by Riemannian gradient descent (Bonnabel 2013), while $\hat\sigma$ has a closed-form estimator.

Once implemented, the Rm-NML framework enables fully coordinate-invariant model selection, regret minimization, and MDL-based coding on manifold-valued data. Notably:
- For hierarchical data or graph embeddings in hyperbolic space, hyperbolic-Gaussian Rm-NML enables selection of both embedding dimension $D$ and curvature (via the geodesic radius $R$), fully respecting the underlying data geometry.
- The framework generalizes to any manifold admitting a Riemannian structure, supporting applications where geometric structure is intrinsic to the data.

A plausible implication is that Rm-NML facilitates rigorous model selection and information-theoretic analyses for emerging applications in geometric deep learning, network analysis, and manifold-based statistical inference, particularly where non-Euclidean geometries are required [2508.21466].

Source: https://www.emergentmind.com/topics/riemannian-nml