---
title: 'BayesSDF: Probabilistic Geometry for 3D Reconstruction'
url: https://www.emergentmind.com/topics/bayessdf
type: topic
---

# BayesSDF: Probabilistic Geometry for 3D Reconstruction

Searching arXiv for BayesSDF and closely related methods.

arxiv_search query: "BayesSDF uncertainty Signed Distance Functions BayesRays NeuS"

arxiv_search({"query":"BayesSDF uncertainty Signed Distance Functions BayesRays NeuS", "max_results": 10, "sort_by": "relevance"})

BayesSDF is a probabilistic uncertainty quantification framework for neural implicit 3D reconstruction in which scene geometry is represented as a Signed Distance Function (SDF) and uncertainty is attached directly to the reconstructed surface rather than only to ray-space radiance or density. Introduced in "A Probabilistic Approach to Uncertainty Quantification Leveraging 3D Geometry" [2507.06269], it is designed to answer where the surface geometry of a reconstruction is unreliable, and by how much, in a computationally tractable manner aligned with actual 3D geometry. Its motivating applications include scientific simulation in complex 3D environments, such as fluid flow through forests, and robotic settings in which explicit surface geometry and awareness of fidelity surface geometric uncertainty are required.

## 1. Concept and problem setting

BayesSDF is built on top of SDF-based neural implicit reconstruction, in practice a NeuS-like model, and combines three ingredients: a neural SDF representation of geometry, a learned deformation field defined over space, and a curvature-like uncertainty estimate derived from the sensitivity of rendered color to local geometric perturbations [2507.06269]. The core idea is to represent scene geometry as a neural SDF, augment it with a deformation field, measure how sensitive rendering is to perturbations of that field, and interpret that sensitivity through a Laplace-inspired Hessian proxy as local geometric uncertainty.

The problem addressed is uncertainty quantification in neural implicit 3D representations, specifically SDFs. Existing methods for NeRF-like volumetric radiance representations tend to estimate uncertainty along rays, produce uncertainty maps in image space that are weakly tied to the underlying surface geometry, and incur substantial compute and memory cost when they rely on ensembles or repeated sampling. BayesSDF instead provides surface-aware uncertainty: uncertainty is tied to local sensitivity of the SDF geometry and its effect on rendering, giving spatially coherent uncertainty on reconstructed surfaces [2507.06269].

The framework is motivated by settings in which geometry is not merely a visual representation but a boundary condition for downstream computation. In scientific simulation, the shape and location of surfaces determine flow patterns, structural stresses, or contact. In robotics, reliable reconstruction is needed not only for perception but also for planning around obstacles, manipulating cluttered scenes, and prioritizing active perception. This suggests that the principal contribution of BayesSDF is not only uncertainty estimation per se, but the redefinition of uncertainty as a geometric property of an implicit surface rather than as an emergent by-product of volumetric rendering.

## 2. Geometric foundations in SDF-based reconstruction

BayesSDF is grounded in the distinction between SDF-based geometry and radiance-based scene representations. Radiance-based models such as NeRF or 3D Gaussian splatting model radiance or density fields, but do not natively provide a signed distance value, a well-defined zero-level set, or analytical surface normals or curvature. By contrast, an SDF explicitly defines geometry through a scalar field
\[
f:\mathbb{R}^3 \to \mathbb{R}
\]
with
\[
f(\mathbf{x}) =
\begin{cases}
\phantom{-}d, & \text{if } \mathbf{x} \text{ is outside the surface} \\
\phantom{-}0, & \text{if } \mathbf{x} \text{ lies on the surface} \\
-d, & \text{if } \mathbf{x} \text{ is inside the surface}.
\end{cases}
\]
The zero level set \(\{\mathbf{x}: f(\mathbf{x})=0\}\) is the surface, and \(\nabla f(\mathbf{x})\) provides normals [2507.06269].

The underlying geometry is embedded in a NeuS-style pipeline. Geometry is represented by an SDF model \(f(\mathbf{x})\), typically parameterized by an MLP, and rendering converts SDF values to opacity via
\[
\alpha(\mathbf{x}) = \Phi_s(f(\mathbf{x})) = \frac{1}{1 + e^{-s f(\mathbf{x})}}
\]
with scale \(s\). Color composition along a ray \(\mathbf{r}(t)=\mathbf{o}+t\mathbf{d}\) is written as
\[
C(\mathbf{r}) = \sum_i T_i \alpha_i \mathbf{c}_i, \quad T_i = \prod_{j=1}^{i-1}(1-\alpha_j),
\]
where \(\mathbf{c}_i\) is the color at sample point \(\mathbf{x}_i\) and \(\alpha_i=\alpha(\mathbf{x}_i)\) [2507.06269].

This geometric explicitness is central to BayesSDF’s uncertainty semantics. Because NeuS sampling concentrates near the SDF zero level set, any local sensitivity analysis of rendering with respect to geometry is naturally concentrated near surfaces. A plausible implication is that BayesSDF’s uncertainty field is not simply projected onto surfaces after the fact; it is constructed in a way that is already biased toward the zero-level set that defines the surface itself.

## 3. Probabilistic formulation and deformation-based uncertainty

BayesSDF introduces a deformation field \(\mathbf{d}(\mathbf{x}) \in \mathbb{R}^3\) that locally distorts geometry and thereby affects rendering:
\[
\mathbf{c}(\mathbf{x}) = \mathcal{R}\bigl(\mathbf{x}, \mathbf{d}(\mathbf{x}); \Theta\bigr),
\]
where \(\mathcal{R}\) is the rendering operator and \(\Theta\) denotes global SDF and appearance parameters [2507.06269]. Implementation-wise, \(\mathbf{d}(\mathbf{x})\) is represented using a single-level hash encoding. The scene bounding box \(\mathbf{aabb}\in\mathbb{R}^6\) normalizes world coordinates to a unit cube, the hash grid has resolution \(2^\ell\) in each dimension, hash table size \(2^{3\ell+1}\), and stores 3D deformation vectors at each grid vertex. For a point \(\tilde{\mathbf{x}}\), the deformation field is obtained by trilinear interpolation across the eight voxel corners:
\[
\mathbf{d}(\mathbf{x}) = \sum_{c=1}^{8} \alpha_c(\tilde{\mathbf{x}})\,\mathbf{d}_c,
\]
with interpolation weights \(\alpha_c \in [0,1]\) satisfying \(\sum_{c=1}^{8}\alpha_c=1\).

The probabilistic motivation is a Laplace approximation. In standard form, a posterior over parameters \(\theta\) around the MAP estimate \(\hat\theta\) is approximated as
\[
p(\theta \mid \mathcal{D}) \approx \mathcal{N}(\hat{\theta}, H^{-1}),
\]
where \(H\) is the Hessian or a Gauss-Newton or Fisher surrogate, and predictive uncertainty at input \(\mathbf{x}\) is approximated by
\[
\mathrm{Var}[f(\mathbf{x})] \approx \nabla_\theta f(\mathbf{x})^\top H^{-1}\nabla_\theta f(\mathbf{x}).
\]
BayesSDF adopts this structure in a specialized form. The parameters of interest are the deformation variables \(\{\mathbf{d}_c\}\), and instead of forming or inverting a full Hessian, the method accumulates a diagonal, curvature-like statistic based on squared gradients of rendered color with respect to those deformation variables [2507.06269].

For each sample point \(\mathbf{x}\) along a ray, the rendered color is \(\mathbf{c}(\mathbf{x})=(r,g,b)\). For each corner deformation vector dimension \(d \in \{1,2,3\}\), BayesSDF computes
\[
\frac{\partial r}{\partial \mathbf{d}_d}, \quad
\frac{\partial g}{\partial \mathbf{d}_d}, \quad
\frac{\partial b}{\partial \mathbf{d}_d},
\]
using automatic differentiation and separate backward passes for each channel. For each grid corner \(k\), it aggregates squared partial derivatives into a scalar curvature proxy:
\[
\Delta(\mathbf{x}) = \sum_{d=1}^{3}
\Bigl(
\frac{\partial r}{\partial \mathbf{d}_d}^2
\cdot
\frac{\partial g}{\partial \mathbf{d}_d}^2
\cdot
\frac{\partial b}{\partial \mathbf{d}_d}^2
\Bigr).
\]
These quantities are accumulated over samples and rays into a 3D tensor \(\mathbf{H}\) of size \(((2^\ell)+1)^3\), treated as a Hessian-like quantity [2507.06269].

The geometric interpretation is local surface instability. For infinitesimal perturbations \(\delta \mathbf{d}(\mathbf{x})\),
\[
\mathbf{c}\bigl(\mathbf{x};\,\mathbf{d}(\mathbf{x})+\delta\mathbf{d}(\mathbf{x})\bigr)
\approx
\mathbf{c}\bigl(\mathbf{x};\,\mathbf{d}(\mathbf{x})\bigr)
+
\nabla_{\mathbf{d}(\mathbf{x})}\mathbf{c}(\mathbf{x})\,\delta\mathbf{d}(\mathbf{x}).
\]
Within a Laplace-like view, if the deformation parameters have Gaussian posterior covariance \(\Sigma_{\mathbf{d}}\), then
\[
\mathrm{Var}[\mathbf{c}(\mathbf{x})]
\approx
\nabla_{\mathbf{d}(\mathbf{x})}\mathbf{c}(\mathbf{x})^\top
\Sigma_{\mathbf{d}}
\nabla_{\mathbf{d}(\mathbf{x})}\mathbf{c}(\mathbf{x}).
\]
BayesSDF simplifies this by treating squared gradients, or a normalized version, directly as a measure of sensitivity and uncertainty. The uncertainty field over space is defined through the Hessian grid as
\[
\sigma(\mathbf{x}) = \sqrt{\mathbf{H}\bigl(\text{grid\_index}(\mathbf{x})\bigr)}.
\]
High \(\sigma(\mathbf{x})\) is interpreted as geometry that is fragile: small changes in deformation parameters produce large changes in appearance, indicating underconstrained geometry [2507.06269].

A common misconception is that BayesSDF computes a full Bayesian posterior over 3D geometry. The method instead uses a Laplace-inspired approximation and a diagonal curvature proxy. The paper’s description has some notation quirks, and the squared-gradient accumulation is heuristic rather than a full Gaussian posterior treatment. This suggests that BayesSDF is best understood as a practical probabilistic surrogate for geometry-aware uncertainty, not as an exact posterior over surfaces.

## 4. Training, inference, and computational characteristics

The underlying SDF or NeuS model is trained in the standard way: rays are sampled from training views; a proposal sampler hierarchically refines sample locations along rays to focus around the SDF’s zero level set; and rendered pixel colors are optimized with a standard photometric reconstruction loss, typically per-pixel \(L_1\) or \(L_2\) between rendered and ground-truth RGB [2507.06269]. The uncertainty machinery is not obtained by training ensembles of models or by repeated Monte Carlo sampling at inference time.

Instead, the Hessian-like grid \(\mathbf{H}\) is computed once, or periodically, after training. The procedure is to load a trained BayesSDF or NeuS model, iterate over rays and sampled positions, compute color and the computational graph linking color to the deformation parameters, backpropagate once per color channel to obtain gradients with respect to grid-corner deformations, square and accumulate the gradients into \(\mathbf{H}\), and zero gradients between channels. After sufficient samples, each grid corner stores a sum of squared partial derivatives [2507.06269].

At inference time, the precomputed \(\mathbf{H}\) tensor is loaded together with the trained reconstruction model. When rendering a novel view, the method samples along each ray using the existing NeuS sampler, locates the grid cell or corner indices associated with each 3D sample, looks up or interpolates the relevant \(\mathbf{H}\) values, converts them to uncertainty via \(\sigma(\mathbf{x})=\sqrt{\mathbf{H}(\text{grid\_index}(\mathbf{x}))}\), and aggregates them to a per-pixel uncertainty map, typically near the dominant surface sample or by a weighted combination along the ray [2507.06269]. The output is therefore an RGB image, a depth image, and an uncertainty image.

Three computational strategies define the method’s efficiency profile. First, the hash-encoded deformation grid gives an efficient parameterization of geometric variability. Second, the Hessian approximation is diagonal and per-vertex rather than dense, so no matrix inversion is required. Third, channel-wise backpropagation limits memory usage. Relative to ensembles or repeated Monte Carlo inference, this is substantially lighter. At the same time, the paper notes that the Hessian accumulation still requires extra backward passes and an additional pass over the dataset, so the offline cost remains nontrivial for very large scenes [2507.06269].

## 5. Evaluation, calibration, and empirical behavior

BayesSDF is evaluated on synthetic and real datasets including “Basket”, “Africa”, “Statue”, and “Torch”, using RGB and depth rendering from trained NeuS-like models together with uncertainty maps and ensemble-based baselines [2507.06269]. The evaluation does not focus on classical calibration scores such as ECE or NLL. Instead, it measures how well uncertainty ranks actual geometric error.

For each rendered image, the per-pixel depth error is
\[
e(u,v)=\left|\hat d(u,v)-d^\star(u,v)\right|.
\]
Pixels are sorted by predicted uncertainty, progressively removed by sparsification, and the Mean Absolute Error on the remaining pixels is computed as
\[
\Delta\text{MAE}(p)=
\frac{1}{|\Omega(p)|}
\sum_{(u,v)\in\Omega(p)}
\bigl|\hat d(u,v)-d^\star(u,v)\bigr|,
\]
where \(\Omega(p)\) is the set of remaining pixels after removing a fraction \(p\). The summary metric is Area Under the Sparsification Error curve (AUSE), where lower values indicate better correlation between uncertainty ranking and true error [2507.06269].

| Dataset | Ensemble | BayesSDF |
|---|---:|---:|
| Basket | 0.206 | 0.218 |
| Africa | 0.201 | 0.268 |
| Statue | 0.150 | 0.148 |
| Torch | 0.164 | 0.182 |

These results show that BayesSDF is competitive with ensembles: slightly worse on Basket and Africa, slightly better on Statue, and close on Torch [2507.06269]. Since BayesSDF uses a single trained model and a one-time Hessian computation, the comparison is significant primarily as an efficiency-performance trade-off rather than as an outright dominance claim.

Qualitative findings further define the method’s behavior. High uncertainty aligns with fine, complex object boundaries, occlusion boundaries, and regions with limited view coverage, while low uncertainty appears on large, well-observed, planar surfaces [2507.06269]. The paper interprets these patterns as confirmation that Hessian-based uncertainty tracks real reconstruction errors rather than arbitrary radiance variance. A plausible implication is that BayesSDF is especially informative in precisely those regions where small geometric misplacements would be most consequential for simulation or control.

## 6. Applications, limitations, and relation to prior work

BayesSDF is positioned for downstream settings in which explicit surfaces and uncertainty are both operational requirements. In physics-based simulation, uncertainty maps can be used to mask or down-weight uncertain geometry regions or to drive adaptive refinement. In robotics, uncertainty can be used to avoid planning through poorly reconstructed obstacles or to focus active perception on ambiguous areas. The paper also identifies medical imaging and scientific modeling as relevant settings for explicit identification of unreliable geometry [2507.06269]. The primary narrative example is forests and other complex natural environments, where one wishes to reconstruct geometry from images and then simulate fluid flow or related processes through uncertain boundaries.

Its limitations are equally central to its interpretation. The Laplace or Hessian treatment is approximate and diagonal; no full posterior covariance is computed. The assumption that high sensitivity of rendered color to deformation corresponds to high geometric uncertainty is plausible but not guaranteed to be perfectly calibrated. Strongly textured regions, for example, could induce high color sensitivity even if geometry is correct. The method targets static scenes, so extension to dynamic scenes would require modeling time-varying geometry. Moreover, although it is cheaper than ensembles, the offline accumulation of Hessian-like statistics is still an additional computational phase [2507.06269].

Within the broader literature, BayesSDF is explicitly contrasted with NeRF and BayesRays-style uncertainty, where uncertainty is defined in ray space as variance of densities or colors along a ray and therefore need not align with actual surface uncertainty. It is also contrasted with ensembles, MC dropout, and generic Bayesian neural networks, which estimate epistemic uncertainty over model parameters without exploiting SDF geometry directly [2507.06269]. BayesSDF’s novelty lies in direct integration of geometric SDF properties into uncertainty estimation, a Hessian-based and Laplace-inspired measure applied to a geometry-focused deformation field rather than generic network weights, and a practical single-model workflow that yields uncertainty competitive with ensembles while remaining surface-aware.

BayesSDF therefore occupies a specific methodological niche: uncertainty quantification for neural implicit geometry in which the uncertainty object is neither an image-space confidence map nor a full posterior over surfaces, but a local, deformation-based measure of how fragile the rendered scene geometry is under small perturbations. In that sense, it establishes a foundation for uncertainty-aware 3D scene reconstruction, simulation, and robotic decision-making [2507.06269].

Source: https://www.emergentmind.com/topics/bayessdf