The Cost Geometry of Belief: finite-resource inference under noisy observation

Abstract: A finite agent, a machine's digital twin, or any bounded reasoner, sees a fixed, noisy world only through finite sensors, so its coherent output is not a point but a belief: a probability density over states (the Bayes posterior). Certainty is denied twice, by observation (Cramer-Rao) and by physics (Landauer), both diverging at the boundary where the Fisher information blows up. We turn this finiteness into geometry: belief-cost geometry, the geometry of what it costs to change one's mind. The cost metric is optimal transport in Wasserstein space, conformally reweighted by Fisher information (the price of the precision at stake), $\tilde g_{e,U}=2(e+U)\,g_{W_2}$ with relief $U$. It rests on two posed postulates: that revision cost is a scalar price on transport (the arena), and that the price is honest, one nat costs the same length everywhere (eikonal). Honesty selects the Fisher reweighting because transport demotes the Fisher information from the metric ruler of distinguishability (its role in Fisher-Rao information geometry) to the slope of entropy. Three results follow on the conformal class: a wall, a well-posed inference pushes certainty to infinite cost-distance once the relief dominates the Fisher information (necessity conjectured beyond power laws); an honest family, the eikonal price is equivalent to $U=cJ$, the Fisher family; and a rigidity (essentially location-scale), these geometries are hyperbolic and the Stam bound crowns the Gaussian as the most hyperbolic location-scale belief (ranking at $e=0$), the value $-1/4$ being one image of a relativity of cost. The cost of reaching a given precision then has a geometric floor diverging at certainty. Thermodynamics fixes the cost unit (one nat costs $k_BT$ at the wall) and motivates the framework; the results are geometric, in nats.

• The paper introduces a cost geometry framework for belief revision that models inference under noisy observation using optimal transport and Fisher information.
• It shows that the cost of updating beliefs is bounded by thermodynamic limits and results in an infinite cost 'wall' that prevents complete certainty.
• The framework has practical implications for robust Bayesian updating and learning algorithms by quantifying the thermodynamic and computational costs of belief revision.

The Cost Geometry of Belief: Finite-Resource Inference Under Noisy Observation

Introduction and Problem Statement

The paper "The Cost Geometry of Belief: finite-resource inference under noisy observation" (2606.21585) presents a comprehensive framework for modeling the cost of belief revision in finite agents. These agents interact with a fixed, noisy world through limited sensors, and consequently, their epistemic state is a Bayesian posterior—a probabilistic belief—rather than a point estimate. The core thesis posits that the process of belief update for such agents is geometrically structured by constraints from both information theory and thermodynamics: information is fundamentally bounded both by the finite nature of observation (Cramér-Rao) and by the physical cost of information processing (Landauer’s principle).

This dual limitation ensures that complete certainty is unreachable—beliefs can only be updated within a space where certainty (i.e., the Dirac delta) recedes to infinite cost. The paper seeks to formalize this constraint through a derived geometry, termed "belief-cost geometry," which quantifies the cost to move (revise) a belief within the space of all plausible beliefs.

Cost Geometry Construction

The construction of the cost geometry is based on two primary postulates:

1. Arena Postulate (Transport Cost): The cost of belief revision is modeled as a scalar field conformally reweighting the optimal transport (Wasserstein-2) metric on the space $P_2(\mathbb{R}^n)$ of probability distributions with finite second moments. This choice is justified by the Riemannian structure required for notions such as curvature and geodesics.
2. Honesty Postulate (Entropic Price): The scalar field (termed relief $U$) encoding the cost per unit knowledge is uniform: one nat costs the same length everywhere—formally, the metric slope of Shannon entropy $-H$ is constant throughout belief space.

These postulates culminate in a canonical metric:

$g_{e,U} = 2\,(e + U)\,g_{W_2}$

where $g_{W_2}$ is the tangent-space metric for Wasserstein-2 (transport), $U$ is the local price of precision (relief), and $e$ is a non-negative baseline. $U$ is further characterized as a function of the Fisher information $J$: specifically, the only honest family is $U = cJ$, with $U$0, a direct consequence of the honesty postulate and the Otto identity $U$1.

Key Results: Wall, Eikonal, and Hyperbolicity

The technical development organizes key results into three nested classes of admissible cost geometries, each with both structural and operational consequences.

The Wall: Certainty at Infinite Cost

The framework introduces the notion of a "wall": certainty (point-mass measures) lies at infinite cost-distance in the conformal geometry as soon as $U$2 dominates $U$3 near the boundary. On the power family $U$4, the condition $U$5 is necessary and sufficient. For $U$6, this is always satisfied, and thus the wall is robustly present. The wall achieves two things:

• Ensures well-posedness: No belief update can reach certainty at a finite cost, and hence inference remains in the interior of the belief space.
• Cost floor: The minimal cost to achieve a given reduction in entropy is strictly positive and diverges at certainty.

Eikonal Family: Honesty and Uniqueness

By demanding a constant metric slope for $U$7, the eikonal (honest pricing) condition uniquely singles out the Fisher family: $U$8. This result is only possible in the conformal reweighting class, leveraging the demotion of Fisher information from a Riemannian metric (as in Fisher-Rao geometry) to the transport-gradient of entropy. Thus, the cost to increase knowledge is directly proportional to the Fisher information—a key bridge to both thermodynamic interpretations (Landauer’s cost per nat) and established notions in Bayesian learning (Fisher as certainty).

Curvature: Hyperbolicity and Stam Rigidity

The cost geometry is shown to be hyperbolic, with sectional curvature

$U$9

on location-scale families, where $-H$0 is the Fisher information of the standardized base. Stam’s inequality shows that among all location-scale families with fixed variance, the Gaussian achieves the minimal Fisher information ($-H$1), hence maximal curvature ($-H$2). This structure is orthogonal to that in Fisher-Rao geometry, where curvature relates to the scale Fisher information and does not have this extremal property.

Theoretical and Practical Implications

The framework delineated in this work carries several implications:

• Algorithmic Cost of Revision: Finite inference procedures (e.g., the Kalman filter, Bayesian ensemble updates) correspond to paths along finite-cost geodesics in belief space. A belief stuck at the wall (certainty) is unrevisable without retraining—mirrored in practices such as retraining overconfident neural networks.
• Energetic Interpretation: The metric assigns a thermodynamic cost (in nats) to belief changes, matched to Landauer’s minimum dissipation—one nat of entropy reduction costs $-H$3 at the wall.
• Geometry of Robustness: The negative curvature induced by Fisher reweighting constrains the possible configurations and transitions within belief space, fostering robustness against overcommitment. The ranking of belief families by curvature provides a geometric index of informativeness/non-Gaussianity, potentially relevant for robust (misspecified) Bayesian inference.
• Uniqueness and Canonical Status: The duality to Čencov’s theorem is explicit—while Fisher-Rao is unique for distinguishability, the presented geometry is conjectured to be canonical for revision cost within its conformal class.

Open Problems and Future Directions

Several conjectures and extensions are articulated:

• Necessity of the Wall: The necessity of boundary domination for the wall is established only for the power family; general necessity remains open.
• Curvature Beyond Location-Scale: The negative curvature property is verified for all 1D location-scale families but is open for more general submanifolds in higher dimensions.
• Background Independence: The background cost metric could potentially be coupled self-consistently to the current belief distribution ("active mass" analogy), opening a route to dynamic geometries.

These avenues point toward an overview between statistical, computational, and physical resource theory constraints, where inference, learning, and thermodynamics are intertwined at a fundamental level.

Conclusion

This work establishes a geometric framework for finite-resource probabilistic inference, where the minimal cost for belief revision arises naturally from optimal transport reweighted by Fisher information. The interplay of information-theoretic and thermodynamic limits ensures the inaccessibility of certainty, a uniform (honest) price of knowledge, and hyperbolicity with extremal properties anchored by the Gaussian. These results ground both practical algorithms for learning under uncertainty and theoretical models of bounded rationality in a unified geometric/thermodynamic paradigm. The further development of this theory promises new connections in Bayesian mechanics, robust inference, and learning under physical constraints.