Fisher-Rao Quantization-Aware Distance (FRQAD)

• The paper introduces FRQAD, a similarity metric that models quantization-induced uncertainty by representing each embedding as a diagonal Gaussian, ensuring a systematic preference for high-precision embeddings.
• It leverages the Fisher–Rao geodesic to jointly penalize differences in both the mean and variance of embeddings, ensuring a monotonic increase in distance with decreased bit-width.
• Empirical tests show FRQAD achieves 100% precision in mixed-precision retrieval, outperforming cosine similarity and standard Fisher–Rao methods in discriminating embedding quality.

Fisher-Rao Quantization-Aware Distance (FRQAD) is a principled similarity metric defined on the Gaussian statistical manifold, explicitly modeling quantization-induced uncertainty for neural embeddings. Developed in the context of local agent memory systems, FRQAD systematically discriminates between embeddings of varying bit-widths by inflating variance according to quantization error and evaluating true Fisher–Rao geodesic distance between the induced Gaussians. This construction guarantees monotonic preference for high-precision embeddings and seamlessly integrates with pipelines requiring mixed-precision nearest-neighbor comparisons (Bhardwaj, 6 Apr 2026).

1. Mathematical Formulation

Given two vector embeddings $θ_i, θ_j ∈ ℝ^d$, each with potentially distinct quantization bit-widths $b_i, b_j \in \{2,4,8,32\}$, FRQAD assigns to each embedding a diagonal Gaussian:

$$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$

Here, $σ_{\text{obs}}^2(m_k)$ is the observation variance (reflecting model or measurement noise), $κ$ is a learned or fixed sensitivity exponent (empirically $κ=1$ suffices for discriminating bit-widths), and $I$ is the $d$-dimensional identity matrix.

The FRQAD between two memories is the exact Fisher–Rao (Atkinson–Mitchell) geodesic between their Gaussians:

$$d_{\text{FRQAD}}(m_i, m_j) = \sqrt{ \sum_{\ell=1}^d \left[ \sqrt{2} \cdot \operatorname{arccosh} \left(1 + \frac{(μ_{i\ell} - μ_{j\ell})^2 + 2(σ_{\text{eff},i} - σ_{\text{eff},j})^2}{4σ_{\text{eff},i} σ_{\text{eff},j}} \right) \right]^2 }$$

This form captures both mean displacement and variance mismatch, with the $\operatorname{arccosh}$ term ensuring strict increase in distance when precision decreases. The monotonicity property under bit-width degradation is thus guaranteed (Bhardwaj, 6 Apr 2026).

2. Theoretical Rationale and Derivation

The core tenet is that quantization error injects additive uncertainty, which is orthogonally represented as increased variance in the embedding's distributional model. Unlike cosine similarity or Mahalanobis distance, which ignore intra-vector reliability, FRQAD leverages the information-geometric structure of the Gaussian manifold, ensuring that both displacement in mean and mismatch in uncertainty are penalized.

• The observation variance $b_i, b_j \in \{2,4,8,32\}$0 is interpreted as the base reliability of the embedding model.
• Quantization noise is modeled as multiplicative variance inflation: a lower $b_i, b_j \in \{2,4,8,32\}$1 yields a higher $b_i, b_j \in \{2,4,8,32\}$2, resulting in a "blurrier" representation and hence, higher distance.
• The Atkinson–Mitchell geodesic explicitly incorporates both types of deviation:
  • $b_i, b_j \in \{2,4,8,32\}$3 penalizes differences in deterministic embedding content.
  • $b_i, b_j \in \{2,4,8,32\}$4 penalizes disparities in uncertainty due to precision.
• The $b_i, b_j \in \{2,4,8,32\}$5 parameter modulates the impact of bit-width scaling; a plausible implication is that $b_i, b_j \in \{2,4,8,32\}$6 can be cross-validated for domain-specific optimality.

This structure is grounded in information geometry, where the Fisher–Rao metric is the unique Riemannian metric invariant under sufficient statistics transforms. For diagonal Gaussians, the Atkinson–Mitchell distance analytically quantifies mismatch in both location and scale (Bhardwaj, 6 Apr 2026).

3. Computation and Algorithmic Procedures

The FRQAD is computed as follows:

1. For each embedding $b_i, b_j \in \{2,4,8,32\}$7 ($b_i, b_j \in \{2,4,8,32\}$8 or $b_i, b_j \in \{2,4,8,32\}$9), compute effective standard deviation:

$$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$0

1. For each dimension $$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$1:
   • Compute $$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$2
   • Compute $$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$3
   • Denominator $$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$4
   • Compute $$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$5
2. Aggregate across dimensions:

$$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$6

To ensure numerical stability, the $$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$7 argument is clamped at $$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$8 (i.e. $$\mathcal{N}(θ_k, σ_{\text{eff},k}^2 I),\qquad \text{where}~σ_{\text{eff},k}^2 = σ_{\text{obs}}^2(m_k) \cdot (32/b_k)^{κ}$$9). The complexity is $σ_{\text{obs}}^2(m_k)$0 per pairwise computation, making it tractable for reranking candidate sets.

4. Empirical Performance and Comparative Analysis

FRQAD was benchmarked using 768-dimensional embeddings from nomic-embed-text-v1.5, with 943 factual entries, each stored as both float32 and 4-bit TurboQuant, totaling 18,840 mixed-precision query-fact comparisons.

The key evaluation metric is "mixed-precision preference": the frequency with which the metric ranks the higher-precision (float32) embedding as closer to the query than its quantized version.

Method	Prefers f32	Precision (%)
Cosine	16,127	85.6
Fisher–Rao (no quant)	13,316	70.7
FRQAD	18,840	100.0

FRQAD achieves perfect 100% precision for this task, while cosine similarity and standard Fisher–Rao fail to systematically penalize embedded noise due to quantization. Spearman correlation ($σ_{\text{obs}}^2(m_k)$1) of top-50 ranks is lower for FRQAD ($σ_{\text{obs}}^2(m_k)$2) compared to cosine ($σ_{\text{obs}}^2(m_k)$3); this reflects FRQAD's deliberate reordering to emphasize mean–variance reliability rather than pure similarity (Bhardwaj, 6 Apr 2026).

5. Theoretical Guarantees and Monotonicity

FRQAD possesses a monotonic degradation property:

• For any $σ_{\text{obs}}^2(m_k)$4, $σ_{\text{obs}}^2(m_k)$5.
• The variance-mismatch term in the Atkinson–Mitchell geodesic ensures that increasing $σ_{\text{obs}}^2(m_k)$6 (i.e., reducing bit-width) results in strictly larger distance for all fixed queries.
• Empirically, this translates to perfect recall for higher-precision memories during retrieval in mixed-precision contexts.

6. Implementation and Operational Aspects

Key implementation notes include:

• FLOP count: On 768-dimensional embeddings, a few thousand FLOPs per comparison.
• Stability: Clamp $σ_{\text{obs}}^2(m_k)$7 arguments; ensure $σ_{\text{obs}}^2(m_k)$8 is not underestimated.
• $σ_{\text{obs}}^2(m_k)$9 parameter can be cross-validated via held-out validation.
• $κ$0 may be set to $κ$1 (unit-norm assumption) or estimated from residual variance.
• Integration: FRQAD is primarily used as a secondary reranking step in k-NN or reciprocal rank fusion (RRF) stages, triggered only when comparing embeddings of different precisions.
• Storage: Embeddings are maintained in float32 until demoted; quantization-aware distances are only evaluated as necessary.

For uniform-precision datasets, FRQAD reduces to the standard Fisher–Rao or Mahalanobis metric and offers no systematic advantage (Bhardwaj, 6 Apr 2026).

7. Applications and Limitations

Applications:

• Mixed-precision vector databases and persistent memory systems managing embeddings of nonuniform bit-widths.
• Local agent memory systems that leverage progressive quantization (lifecycling old or less salient memories) yet require reliable retrieval and discrimination.
• Retrieval and clustering in domains where quantization uncertainty must be explicitly incorporated into similarity computations.

Limitations:

• The diagonal Gaussian assumption precludes off-diagonal covariance modeling within embeddings.
• Higher computational overhead than cosine similarity; advisable as a filtering/reranking stage rather than a global search metric.
• Performance is contingent on well-calibrated $κ$2; improper scaling can result in skewed distance scales.
• For purely uniform-precision settings, the metric reverts to standard information-geometric comparisons, yielding minimal benefit relative to simpler alternatives (Bhardwaj, 6 Apr 2026).

8. Related Work and Extensions

The theoretical underpinnings of FRQAD connect to more general interpolating transport distances between optimal transport ($κ$3) and Fisher–Rao metrics, as developed by Chizat et al. (Chizat et al., 2015). The dynamic formulation allows for both mass transport and mass creation/annihilation, penalized by a Fisher–Rao–weighted source term, and interpolates between $κ$4 (transport-dominated) and pure Fisher–Rao (variance-dominated) as the penalty parameter varies. Existence and uniqueness theorems hold, and practical algorithms based on first-order proximal splitting are available for the numerically efficient computation of FRQAD in the context of measure transport, with extensions to mixtures of Diracs and clustering. This suggests that further generalizations of quantization-aware metrics may adapt optimal transport frameworks for multi-modal or structured embedding spaces (Chizat et al., 2015).