Continuous-state and continuous-action extension of the visitation geometry

Establish a full measure-theoretic extension of the resetting visitation manifold and its dually flat information geometry to continuous state and action spaces.

Background

The paper develops the dually flat geometry of resetting visitation measures for finite state and action spaces. It notes that extending this construction to general continuous spaces requires a measure-theoretic treatment, although finite-dimensional exponential families are already known to form dually flat submanifolds of spaces of measures.

The authors identify this continuous-space extension as an unresolved mathematical challenge, distinct from the finite-dimensional exponential-family case treated later in the appendix.

References

A full measure-theoretic extension to continuous $\mathcal S,\mathcal A$ in the sense of is an open challenge, but exponential families are known to form finite-dimensional dually flat submanifolds of the space of measures.

— The Dually Flat Geometry of Planning as Inference  (2609.04005 - Milosevic et al., 3 Sep 2026) in Section 2, paragraph “Infinite-dimesnsional and Wasserstein geometries”

Let $\mathcal S,\mathcal A$ be standard Borel and $\Pi_T={\pi_w\propto\exp\langle w,T\rangle}$ an exponential-family policy class with sufficient statistic $T\in L2$ and full-support restart law $\mu$. Under the regularity conditions of parametrized measure models ($k$-integrability of $T$, $\sigma$-finiteness of the occupancy) and with the conditional negentropy taken as Bregman potential in the directional-derivative sense of measure-space mirror descent, the resetting visitation measures ${\nu{\pi_w}:w\in\mathbb R{d'}}$ form a finite-dimensional dually flat submanifold of $\mathcal P(\mathcal S\times\mathcal A)$, with $w\mapsto\log\pi_w$ and $w\mapsto\mathbb E_{\nu{\pi_w}}[T]$ the Legendre-dual coordinates and the compatible-features Fisher matrix as metric.

— The Dually Flat Geometry of Planning as Inference  (2609.04005 - Milosevic et al., 3 Sep 2026) in Appendix, Section “Linear MDPs and log-linear policies,” Conjecture 1 (equation \ref{conj:continuous})

Whether neural circuits realize this geometry, and how it extends to continuous spaces, we leave to future work.

— The Dually Flat Geometry of Planning as Inference  (2609.04005 - Milosevic et al., 3 Sep 2026) in Conclusion