Fisher Information Rate: Methods & Applications

• FIR is a context-dependent concept that quantifies information accumulation via distinct rate-like measures—per-sample in symmetric estimation, dissipation under Gaussian heat flow, and per-symbol in stochastic processes.
• In symmetric mean estimation, smoothing a density with a Gaussian kernel yields an effective Fisher information that enables subgaussian error bounds closely approximating the Cramér–Rao limit.
• FIR also captures geometric complexity in latent diffusion through the squared Hessian of log-density and informs asymptotic learnability in time series, while its active-learning variant (Fisher Information Ratio) serves as a matrix criterion.

Fisher Information Rate (FIR) is an overloaded term whose precise meaning depends on the inferential or dynamical setting. In recent work, it denotes at least three distinct rate-like quantities derived from Fisher information: an effective per-sample information controlling finite-sample location estimation under symmetry; the rate at which Fisher information dissipates under Gaussian heat flow; and the asymptotic per-symbol Fisher information available for learning parameters of a stochastic process from a long correlated sequence. A separate literature uses the same acronym for Fisher Information Ratio, a matrix-trace criterion for active learning, rather than a rate in time, noise scale, or sample size (Gupta et al., 2023, Gu et al., 3 Apr 2026, Riechers, 2023, Sourati et al., 2016).

1. Definitions and terminological scope

For a one-dimensional location family $\{f_\mu(x)=f(x-\mu)\}$ with score $s(x)=f'(x)/f(x)$, the classical Fisher information is

$$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$

and, under mild regularity, also $\mathcal I=-\mathbb E_f[s'(X)]$ (Gupta et al., 2023). This is the reference object from which several FIR notions are derived.

In the mean-estimation setting of symmetric distributions, FIR refers to the effective per-sample information quantity $\mathcal I_r$, the Fisher information of a Gaussian-smoothed density $f_r=f*\phi_r$, with $\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$. The associated score is $s_r=f_r'/f_r$, and

$$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$

Here the rate language is operational: $\mathcal I_r$ is the per-sample information governing high-probability location error via a variance proxy $s(x)=f'(x)/f(x)$0 (Gupta et al., 2023).

In diffusion geometry, the notation changes. For a heat-smoothed measure $s(x)=f'(x)/f(x)$1 with density $s(x)=f'(x)/f(x)$2 and score $s(x)=f'(x)/f(x)$3, Fisher information is

$s(x)=f'(x)/f(x)$4

while Fisher Information Rate is

$s(x)=f'(x)/f(x)$5

In this usage, FIR is literally a dissipation rate along the heat trajectory (Gu et al., 3 Apr 2026).

For stationary stochastic processes, the relevant object is the Fisher information matrix $s(x)=f'(x)/f(x)$6 of the length-$s(x)=f'(x)/f(x)$7 sequence probability $s(x)=f'(x)/f(x)$8, with increment

$s(x)=f'(x)/f(x)$9

and asymptotic FIR

$$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$0

This is a per-symbol information rate: if $$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$1, then the Cramér–Rao lower bound scales as $$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$2 (Riechers, 2023).

The acronym is therefore context-sensitive.

Usage	Formal object	Setting
Per-sample FIR	$$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$3	Symmetric mean estimation
Dissipative FIR	$$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$4	Heat flow, diffusion geometry
Process FIR	$$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$5	Stochastic processes, time series
Fisher Information Ratio	$$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$6	Active learning

2. FIR as effective per-sample information in symmetric mean estimation

In finite-sample symmetric mean estimation, the central claim is that symmetry permits location estimation at a subgaussian-type rate controlled by smoothed Fisher information rather than by variance alone (Gupta et al., 2023). Let $$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$7 be symmetric about its mean $$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$8, with variance $$\mathcal I(f)=\mathbb E_{X\sim f}[s(X)^2] =\int_{\mathbb R}\Big(\frac{f'(x)}{f(x)}\Big)^2 f(x)\,dx,$$9, and let

$\mathcal I=-\mathbb E_f[s'(X)]$0

For any smoothing radius $\mathcal I=-\mathbb E_f[s'(X)]$1 satisfying $\mathcal I=-\mathbb E_f[s'(X)]$2, the estimator $\mathcal I=-\mathbb E_f[s'(X)]$3 constructed in the paper obeys

$\mathcal I=-\mathbb E_f[s'(X)]$4

Equivalently,

$\mathcal I=-\mathbb E_f[s'(X)]$5

The bound is nonasymptotic and matches the known-$\mathcal I=-\mathbb E_f[s'(X)]$6 location-model form up to smoothing and small multiplicative factors.

The estimator is a three-phase sample-splitting construction. First, an initial robust location estimate $\mathcal I=-\mathbb E_f[s'(X)]$7 is computed on about $\mathcal I=-\mathbb E_f[s'(X)]$8 samples via the median of pairwise means,

$\mathcal I=-\mathbb E_f[s'(X)]$9

Second, another $\mathcal I_r$0 samples are used to build a Gaussian KDE

$\mathcal I_r$1

its score $\mathcal I_r$2, and a clipped, antisymmetrized score $\mathcal I_r$3, with clipping threshold scaling like

$\mathcal I_r$4

A plug-in information estimator is then defined by

$\mathcal I_r$5

Third, on the remaining samples, fresh noise $\mathcal I_r$6 is added, $\mathcal I_r$7, and a one-step Newton refinement is performed: $\mathcal I_r$8

The role of smoothing is dual. It regularizes the unknown density so that the score is well defined and concentration-friendly, and it interpolates between finite-sample robustness and asymptotic efficiency. As $\mathcal I_r$9, $f_r=f*\phi_r$0; for “nice” symmetric densities such as Laplace,

$f_r=f*\phi_r$1

Since the recommended scale is $f_r=f*\phi_r$2, the resulting bound is within a multiplicative $f_r=f*\phi_r$3 of the Cramér–Rao-optimal asymptotic rate $f_r=f*\phi_r$4.

The paper emphasizes that symmetry is essential. Without symmetry and with unknown $f_r=f*\phi_r$5, variance-based subgaussian bounds of order $f_r=f*\phi_r$6 are instance-optimal, and Fisher-information-style improvement is impossible in general. Under symmetry, antisymmetrized scores let the estimator depend on the local sharpness of $f_r=f*\phi_r$7 near its mean rather than only on global variance. For the Gaussian model, $f_r=f*\phi_r$8 and $f_r=f*\phi_r$9; for Laplace with variance $\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$0, $\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$1, yielding a factor-of-2 asymptotic improvement over $\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$2; for Cauchy, $\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$3, showing that heavy tails do not preclude adaptation once symmetry and smoothing are available.

3. FIR as Fisher-information dissipation under heat flow and latent geometry

In latent diffusion analysis, FIR is defined as the rate at which Fisher information decreases as Gaussian noise variance $\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$4 increases (Gu et al., 3 Apr 2026). If $\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$5 and $\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$6 is its density, then

$\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$7

where $\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$8. FIR is therefore the expected squared Hessian of the log-density, or equivalently the Frobenius energy of the score Jacobian. This makes it a second-order geometric functional.

The paper’s central decomposition connects FIR to denoising difficulty through the I–MMSE identity: $\phi_r(x)=\frac{1}{\sqrt{2\pi}r}\exp(-x^2/(2r^2))$9 and thus

$s_r=f_r'/f_r$0

The FI term is controlled by first-order geometry, while the FIR term is controlled by second-order local geometry. In this framework, denoising resistance grows linearly with $s_r=f_r'/f_r$1 and FI, and quadratically with $s_r=f_r'/f_r$2 times FIR.

A key result is the separation between global isometry and local curvature. If an encoder $s_r=f_r'/f_r$3 is globally isometric on tangents, $s_r=f_r'/f_r$4 for all $s_r=f_r'/f_r$5, then $s_r=f_r'/f_r$6: global isometry aligns FI. FIR, however, depends on the transformed Hessian,

$s_r=f_r'/f_r$7

so nonlinear encoders inject curvature directly through $s_r=f_r'/f_r$8.

Under flat-support assumptions, the paper decomposes latent FIR degradation into three measurable penalties: dimensional compression, tangential distortion, and curvature injection. Dimensional compression contributes an exact normal-direction term $s_r=f_r'/f_r$9 in ambient space and $$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$0 in latent space. Tangential distortion is controlled by near-isometry bounds such as $$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$1. Curvature injection appears through a Taylor residual $$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$2, with nonlinear stability bounds of the form

$$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$3

when $$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$4, and an additional dimensional term when $$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$5. The factor $$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$6 shows that smaller $$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$7 exposes more second-order encoder curvature.

The practical estimators mirror the theory. FI is estimated from score norms,

$$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$8

and FIR is estimated by Hutchinson’s trick with JVPs,

$$\mathcal I_r=\mathbb E_{f_r}[s_r(X)^2]=-\mathbb E_{f_r}[s_r'(X)].$$9

using Rademacher probes, or by finite-difference directional derivatives. Typical sample sizes are $\mathcal I_r$0 and $\mathcal I_r$1, with numerical instability at very small $\mathcal I_r$2.

Experimentally, FIR tracks latent diffusability more sharply than reconstruction error alone. On FFHQ $\mathcal I_r$3, GPE and VAE latents with $\mathcal I_r$4 show FI/FIR curves for GPE consistently closer to the pixel baseline; the empirical bi-Lipschitz constants are $\mathcal I_r$5, $\mathcal I_r$6 for GPE and $\mathcal I_r$7, $\mathcal I_r$8 for VAE. Scaled FIR deviation is substantially smaller for GPE at small $\mathcal I_r$9. The generation metrics remain poor relative to pixel diffusion, but the geometry-based FIR diagnostics differentiate encoder quality in a way not captured by reconstruction MSE alone.

4. FIR as asymptotic per-symbol information in stochastic processes

For parametric stochastic processes observed as time series, FIR is the asymptotic increment of sequence Fisher information with respect to observation length (Riechers, 2023). If $s(x)=f'(x)/f(x)$00 is the Fisher information matrix of $s(x)=f'(x)/f(x)$01, then

$s(x)=f'(x)/f(x)$02

and the asymptotic FIR is

$s(x)=f'(x)/f(x)$03

When the limit exists, $s(x)=f'(x)/f(x)$04, where $s(x)=f'(x)/f(x)$05 is the finite-length correction and $s(x)=f'(x)/f(x)$06 is the excess information. The Cramér–Rao lower bound then yields asymptotic estimator variance of order $s(x)=f'(x)/f(x)$07.

A notable feature is the closed form available for any valid unifilar presentation. If $s(x)=f'(x)/f(x)$08 is the recurrent-state set, $s(x)=f'(x)/f(x)$09 its stationary distribution, and $s(x)=f'(x)/f(x)$10 the next-symbol probabilities, then

$s(x)=f'(x)/f(x)$11

This holds even for processes of infinite Markov order, provided the unifilar presentation is valid in a neighborhood of $s(x)=f'(x)/f(x)$12. In computational-mechanics terms, the $s(x)=f'(x)/f(x)$13-machine is the minimal such presentation when its topology is $s(x)=f'(x)/f(x)$14-stable.

Finite-$s(x)=f'(x)/f(x)$15 behavior is governed by the observation-induced metadynamics over estimation states. If $s(x)=f'(x)/f(x)$16 denotes the labeled transition operators and $s(x)=f'(x)/f(x)$17, then the information vector $s(x)=f'(x)/f(x)$18 satisfies

$s(x)=f'(x)/f(x)$19

With $s(x)=f'(x)/f(x)$20 the spectral projection onto eigenvalue $s(x)=f'(x)/f(x)$21 and $s(x)=f'(x)/f(x)$22, one obtains

$s(x)=f'(x)/f(x)$23

This representation yields exact finite-length CRLBs and exposes how convergence depends on the spectrum of $s(x)=f'(x)/f(x)$24.

The convergence modes are structurally rich. Zero eigenvalues generate ephemeral contributions supported only up to a finite index. Eigenvalues with $s(x)=f'(x)/f(x)$25 generate exponential or polynomial-exponential decay. Unit-modulus eigenvalues $s(x)=f'(x)/f(x)$26 generate persistent oscillations. The same spectral data govern the convergence of the myopic entropy rate to the Shannon entropy rate, so the relaxation timescales for myopic information and myopic entropy coincide exactly.

The examples are varied. For an IID biased coin, $s(x)=f'(x)/f(x)$27, $s(x)=f'(x)/f(x)$28 for all $s(x)=f'(x)/f(x)$29, and $s(x)=f'(x)/f(x)$30. For finite-Markov-order families such as the $s(x)=f'(x)/f(x)$31–$s(x)=f'(x)/f(x)$32 Golden Mean process,

$s(x)=f'(x)/f(x)$33

and $s(x)=f'(x)/f(x)$34 for all $s(x)=f'(x)/f(x)$35, so saturation occurs at the cryptic order. For the infinite-Markov-order Even process,

$s(x)=f'(x)/f(x)$36

while $s(x)=f'(x)/f(x)$37 oscillates and decays exponentially, with excess information

$s(x)=f'(x)/f(x)$38

Other examples include the Teddy Bear process, the Simple Nonunifilar Source with a rapidly convergent infinite series for $s(x)=f'(x)/f(x)$39, processes with no finite HMM, and nonergodic mixtures where large $s(x)=f'(x)/f(x)$40 alone does not guarantee efficient learning.

5. Related rate notions in Gaussian convergence and small-noise diffusions

A related but distinct rate problem concerns convergence of Fisher information to Gaussian benchmarks. In the CLT regime, if $s(x)=f'(x)/f(x)$41 for i.i.d. mean-zero variables with smooth density, the standardized Fisher information is

$s(x)=f'(x)/f(x)$42

Under the spectral condition $s(x)=f'(x)/f(x)$43, the paper proves

$s(x)=f'(x)/f(x)$44

and the strengthened monotonicity result that $s(x)=f'(x)/f(x)$45 is non-increasing in $s(x)=f'(x)/f(x)$46 (Johnson, 2019). It also gives the lower bound

$s(x)=f'(x)/f(x)$47

The paper does not name a Fisher Information Rate, but it explicitly identifies a natural CLT-FIR quantity,

$s(x)=f'(x)/f(x)$48

with bounds

$s(x)=f'(x)/f(x)$49

This notion is specific to convergence toward Gaussianity for i.i.d. sums and is separate from process FIR.

For small-noise SDEs, the relevant object is again not a classical FIR but a Fisher-information distance to a Gaussian limit (Dung et al., 2024). For

$s(x)=f'(x)/f(x)$50

with noiseless path $s(x)=f'(x)/f(x)$51 and normalized fluctuation $s(x)=f'(x)/f(x)$52, the Gaussian limit has variance

$s(x)=f'(x)/f(x)$53

Under the stated smoothness and negative-moment assumptions,

$s(x)=f'(x)/f(x)$54

For additive functionals $s(x)=f'(x)/f(x)$55 with fluctuation $s(x)=f'(x)/f(x)$56, the analogous Gaussian limit variance is

$s(x)=f'(x)/f(x)$57

and

$s(x)=f'(x)/f(x)$58

Matching lower bounds show that $s(x)=f'(x)/f(x)$59 is the optimal order in general.

These results do not compute a per-unit-time FIR in the stochastic-process sense. A plausible implication is that, in small-noise regimes, the Fisher-information structure of the diffusion is captured by the Gaussian linearization up to quadratic error in $s(x)=f'(x)/f(x)$60.

6. Conceptual distinctions, limitations, and the active-learning ambiguity

The most persistent source of confusion is terminological. In active learning, FIR means Fisher Information Ratio, not Fisher Information Rate (Sourati et al., 2016). The central quantity is

$s(x)=f'(x)/f(x)$61

where $s(x)=f'(x)/f(x)$62 is a query distribution and $s(x)=f'(x)/f(x)$63 is a test distribution. Under the paper’s type-II discriminative-model assumptions and MLE asymptotics, this quantity upper-bounds the expected variance of the asymptotic log-likelihood ratio: $s(x)=f'(x)/f(x)$64 Thus minimizing $s(x)=f'(x)/f(x)$65 is an inference-oriented active-query objective, but it is not a temporal, per-noise, or per-sample rate.

Across the genuine rate interpretations, the controlling variable differs. In symmetric mean estimation, FIR is indexed by smoothing radius $s(x)=f'(x)/f(x)$66 and enters error bounds through $s(x)=f'(x)/f(x)$67. In diffusion geometry, FIR is indexed by noise variance $s(x)=f'(x)/f(x)$68 and measures dissipation of Fisher information under heat flow. In stochastic processes, FIR is indexed by sequence length and is the long-run increment of $s(x)=f'(x)/f(x)$69. The CLT and small-noise SDE literatures add adjacent notions: rates of convergence of Fisher information or Fisher-information distance toward Gaussian limits.

The limitations are likewise setting-specific. Symmetric mean estimation requires symmetry and uses a smoothing scale $s(x)=f'(x)/f(x)$70 chosen from $s(x)=f'(x)/f(x)$71; removing dependence on prior scale control is identified as an open question. Latent-diffusion FIR preservation theorems assume flat support to isolate encoder-induced curvature, smooth $s(x)=f'(x)/f(x)$72 encoders, and numerically stable score estimation. Process FIR requires stationarity, regularity, and a unifilar presentation valid in a neighborhood of the parameter; nonergodic mixtures require special care because long single sequences do not replace many independent samples. The active-learning ratio framework is asymptotic, presumes identifiability and positive definiteness of the Fisher matrices, and may be sensitive to covariate shift and plug-in approximation.

Taken together, these literatures establish FIR as a family of second-order information descriptors rather than a single invariant object. Depending on context, it quantifies local efficiency for robust location estimation, geometric complexity under diffusion, asymptotic learnability of correlated processes, or—under a conflicting acronym—a matrix criterion for active learning. The common thread is that each formulation refines variance-based reasoning by using derivatives of log-density, conditional transition structure, or Fisher-matrix transport to characterize how information is accumulated, dissipated, or operationalized in inference.