---
title: Localization Uncertainty-Guided Registration
url: https://www.emergentmind.com/topics/localization-uncertainty-guided-registration
type: topic
---

# Localization Uncertainty-Guided Registration

Localization uncertainty-guided registration denotes registration procedures in which an estimated uncertainty over pose, displacement, correspondences, or deformation parameters actively shapes the alignment step rather than being reported only after optimization. In U-ViLAR, the term refers to a module that begins with a coarse camera pose prior $\mathbf P_0=(x_0,y_0,\varphi_0)$, models the remaining ambiguity as a 3-DoF probability distribution, and uses that distribution to guide a local, fine-grained, differentiable registration in Bird’s-Eye-View (BEV) space [2507.04503]. Closely related formulations appear in neuroimaging, where per-voxel Gaussian uncertainty is propagated to global transformation models and used for uncertainty-aware fitting, and in unsupervised medical image registration, where heteroscedastic image uncertainty reduces the influence of regions with high uncertainty during training [2410.09299] [2312.00836]. In LiDAR and landmark-based registration, analogous mechanisms use local information matrices, predicted ICP error covariance, or Bayesian posteriors to constrain ill-conditioned directions, reweight observations, or quantify deformation variability [2211.16335] [1605.09276].

## 1. Problem setting and sources of uncertainty

Registration is repeatedly described in the literature as an ambiguous estimation problem whose failure modes are strongly domain-dependent. In urban visual localization, even after perceptual-uncertainty-aware association, the coarse pose estimate $(x_c,y_c,\varphi_c)$ may remain uncertain or multi-modal because of symmetric street layouts or occlusion; the refinement stage must therefore search locally around an uncertain coarse hypothesis rather than trust a single pose estimate [2507.04503]. In unsupervised medical image registration, the standard homoscedastic assumption implicit in mean-squared-error objectives is often violated because medical images exhibit heteroscedastic and input-dependent noise; this can cause spurious gradients from noise-induced outliers and degrade displacement estimation [2312.00836].

In geometric localization, uncertainty also arises from lack of observability rather than from measurement noise alone. X-ICP characterizes LiDAR-challenging environments as settings in which geometrically uninformative scans deteriorate point cloud registration performance and push optimization toward divergence along weakly constrained directions; the corresponding uncertainty is expressed as fine-grained localizability of individual pose degrees of freedom [2211.16335]. A related but distinct source appears in building-level LiDAR-to-model alignment, where L2M-Reg explicitly addresses the generalization uncertainty in semantic 3D city models at the Level of Detail 2 (LoD2) through reliable plane correspondence establishment, a pseudo-plane-constrained Gauss-Helmert model, and adaptive estimation of vertical translation [2509.16832].

Earlier work in landmark-based image registration emphasized that even small changes in landmark positions can produce large changes in the resulting diffeomorphism. Marsland and Shardlow therefore introduced a Langevin formulation for small random perturbations of Hamiltonian landmark registration, making localization noise a first-class variable in the prior and posterior over deformations [1605.09276]. Taken together, these formulations show that “localization uncertainty” can refer to uncertainty in a coarse global pose, in voxel-wise correspondence coordinates, in deformation parameters, or in the conditioning of the registration objective itself.

## 2. Mathematical representations of localization uncertainty

A central design choice is how the uncertainty is parameterized. In U-ViLAR, the registration stage is defined on a discrete grid of candidate poses
\[
\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},
\]
and a pose decoder outputs three independent discrete distributions $\mathbf p_x\in\mathbb R^{N_x}$, $\mathbf p_y\in\mathbb R^{N_y}$, and $\mathbf p_\varphi\in\mathbb R^{N_\varphi}$. Their per-dimension entropies
\[
U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},
\]
quantify uncertainty in each degree of freedom, while the joint prior over the 3-DoF grid is the product
\[
\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.
\]
This representation is discrete, differentiable, and naturally tied to a local search volume [2507.04503].

In Gaussian-process registration, Luo et al. represent transformation uncertainty by the posterior covariance of a nonrigid deformation field. For each scalar component $d(x)$ of the vector-valued deformation $u(x)$, the prior is $p(d)=GP(m(x),k(x,x'))$ with $m(x)=0$ and $k(x,x')=\exp(-\|x-x'\|^2/a)$. Given observed landmark displacements, the GP posterior at new locations has mean $\mu_*=K_*^\top K^{-1}D$ and covariance $\Sigma_*=K_{**}-K_*^\top K^{-1}K_*$. Point-wise transformation uncertainty is then
\[
\sigma(x)=\sqrt{\Sigma_*(x,x)},
\]
and point-wise registration error is
\[
e(x)=\|d_g(x)-d_*(x)\|_2.
\]
Here uncertainty is attached directly to the predicted displacement field rather than to a local search distribution [1908.07709].

Learning-based neuroimaging work adopts a per-voxel Gaussian model. The network predicts a mean $\mu_{n,j}$ and standard deviation $\sigma_{n,j}$ for each voxel $n$ and direction $j\in\{1,2,3\}$,
\[
p(d_{n,j}\mid I,\theta)=\mathcal N(d_{n,j};\,\mu_{n,j},\,\sigma_{n,j}^2),
\]
leading to the Gaussian-negative-log-likelihood
\[
L_{\mathrm{uncer}}
=
\sum_{n=1}^N\sum_{j=1}^3
\frac{1}{2}\Bigl[
\frac{(d_{n,j}-\mu_{n,j})^2}{\sigma_{n,j}^2}
+\ln(\sigma_{n,j}^2)
\Bigr].
\]
These local variances are then lifted to global parameter uncertainty through weighted least squares and covariance propagation [2410.09299].

A path-space variant appears in the landmark framework of Marsland and Shardlow. Starting from Hamiltonian geodesic registration, they introduce a generalized Langevin SDE on momenta and landmark positions,
\[
\begin{aligned}
dp_i &= \bigl[-\,\lambda\,\nabla_{p_i}H(p,q)-\nabla_{q_i}H(p,q)\bigr]\,dt+\sigma\,dW_i(t),\\
dq_i &= \nabla_{p_i}H(p,q)\,dt,
\end{aligned}
\]
with $\sigma^2=2\lambda/\beta$. The invariant Gibbs density is proportional to $e^{-\beta H(p,q)}$, so localization uncertainty is represented as a thermally perturbed prior over diffeomorphic registrations [1605.09276].

## 3. Mechanisms by which uncertainty guides registration

Once uncertainty is represented, the key methodological question is how it enters the registration objective. In U-ViLAR, each candidate pose $\theta_{i,j,k}$ produces a raw feature-difference cost
\[
D_{\mathrm{cost}(i,j,k)}
=
\Bigl\|
\mathcal T_{\theta_{i,j,k}}(\mathbf F_v^{\mathrm{BEV}})
-\mathbf F_m^{\mathrm{BEV}}
\Bigr\|_2.
\]
This cost is fused with the learned uncertainty prior as
\[
D_{\mathrm{fused}(i,j,k)}
=
D_{\mathrm{cost}(i,j,k)}+\lambda\,\mathbf P_{\mathrm{uncert}(i,j,k)},
\]
then processed by a small 3D CNN to produce a final cost volume $\mathbf C(i,j,k)$. The refined pose is obtained by a 3D softmax weighted average,
\[
(x_f,y_f,\varphi_f)
=
\sum_{i,j,k}
\sigma\!\bigl(\mathbf C+\gamma\,\mathbf P_{\mathrm{uncert}\bigr)_{i,j,k}
\cdot \theta_{i,j,k},
\]
with learned scalars $\lambda$ and $\gamma$. The entire refinement is differentiable, and no external iterative solver or hard PnP step is used [2507.04503].

In heteroscedastic medical image registration, uncertainty guidance takes the form of adaptive residual weighting. The fixed image is modeled under a zero-mean heteroscedastic Gaussian noise model,
\[
p(I_f(x)\mid u,I_m)=\mathcal N(I_f(x);\,I_m(x+u(x)),\,\sigma_I^2(x)),
\]
which yields the heteroscedastic NLL
\[
\mathcal L_{\mathrm{NLL}}(u,\sigma_I)
=
\sum_{x\in\Omega}
\Bigl[
\frac{\|I_f(x)-I_m(x+u(x))\|^2}{\sigma_I^2(x)}
+\log \sigma_I^2(x)
\Bigr]
+\lambda\!\int\!\|\nabla u(x)\|^2\,dx.
\]
Because direct joint optimization of the displacement and variance estimators proved unstable, the framework uses a collaborative or alternating training schedule, with a warm-up stage followed by alternating displacement and variance steps. The displacement loss uses a relative SNR weight
\[
w(x)=\mathcal T\Bigl[\bigl(I_f(x)/\lfloor\hat\sigma_I(x)\rfloor\bigr)^{2\gamma}\Bigr],
\]
so that high-noise, low-SNR regions are down-weighted. Blocking gradients on $\hat\sigma_I$ prevents the displacement network from inflating the variance estimate to suppress its own loss [2312.00836].

In hierarchical neuroimaging registration, uncertainty guidance is expressed as uncertainty-aware fitting of a global transformation model. If $\hat\mu_j=\Phi c_j$ and $W_j=\mathrm{diag}(\sigma_{1,j}^{-2},\dots,\sigma_{N,j}^{-2})$, the weighted least-squares fit is
\[
c_j^\mu
=
(\Phi^T W_j\Phi)^{-1}\Phi^T W_j\mu_j,
\qquad
\Sigma_{c_j}
=
(\Phi^T W_j\Phi)^{-1}.
\]
The optimization is equivalent to minimizing
\[
E_j(c_j)
=
(\Phi c_j-\mu_j)^T W_j(\Phi c_j-\mu_j)+\lambda\,c_j^T R\,c_j.
\]
Here uncertain voxels contribute less to global fitting, and the resulting coefficient covariance supports posterior sampling for downstream propagation [2410.09299].

In LiDAR registration, uncertainty guidance is often implemented as localizability-aware constraints. X-ICP computes the local information matrix
\[
H=\sum_{i=1}^N J_i^\top J_i,
\]
performs the eigenvalue decomposition $H=U\Lambda U^\top$, and declares a principal direction $u_k$ degenerate when the normalized strength
\[
\ell_k=\frac{\lambda_k}{\sum_{j=1}^6\lambda_j}
\]
falls below a threshold $\tau_{\mathrm{deg}}$. The constrained Gauss-Newton step enforces
\[
U_{\mathrm{deg}}^\top\,\delta\xi=0,
\]
or equivalently projects the update into the well-constrained subspace. LP-ICP generalizes this strategy by combining point-to-line and point-to-plane residuals, computing per-direction localizability contributions from Hessian blocks, and adding soft constraints for partially localizable directions together with hard constraints for non-localizable ones [2211.16335] [2501.02580].

A complementary mechanism is to predict the uncertainty of a registration module before or during state estimation. In deep ICP covariance estimation, a network predicts a $6\times6$ SPD covariance $\Sigma_k$ for each LiDAR scan. In an EKF on $SE(3)$, the ICP residual is
\[
r_k=\log\bigl((T_k^-)^{-1}\widehat T_k^{\mathrm{ICP}}\bigr)\in\mathbb R^6,
\]
the innovation covariance is
\[
S_k=H\,P_k^-\,H^\top+\Sigma_k^{\mathrm{(world)}},
\]
and the Kalman gain uses the predicted registration covariance directly. In this setting, uncertainty does not only annotate the ICP output; it changes the filter update that follows the registration [2509.18954].

At building level, L2M-Reg uses an uncertainty-aware plane-based fine registration pipeline consisting of reliable plane correspondence establishment, a pseudo-plane-constrained Gauss-Helmert model, and adaptive estimation of vertical translation, explicitly accounting for model uncertainty in semantic 3D city models [2509.16832].

## 4. Representative systems and reported performance

Reported benefits are heterogeneous because the underlying tasks differ, but the empirical pattern is that uncertainty enters as a useful control signal when it is tightly coupled to the registration mechanism rather than appended as a post hoc score. Representative results span BEV visual localization, medical image registration, brain MRI registration, and LiDAR localization [2507.04503] [2312.00836] [2410.09299] [2108.06771] [2211.16335] [2509.18954].

| System | Domain | Reported result |
|---|---|---|
| U-ViLAR [2507.04503] | nuScenes + HD map | Including LU-Guided Registration reduces orientation MAE on nuScenes from ~0.116° to 0.075° |
| Heteroscedastic framework [2312.00836] | ACDC | VoxelMorph/MSE: 80.20%; VoxelMorph + NLL: 76.49%; VoxelMorph + β-NLL: 78.74%; Ours (γ=0.5): 80.73% |
| Hierarchical uncertainty estimation [2410.09299] | ABIDE / OASIS3 | Affine Dice 0.718→0.730 and 0.673→0.682; B-spline (10 mm) 0.782→0.790 and 0.750→0.772 |
| NPBDREG [2108.06771] | Brain MRI | Dice score 0.74 vs. 0.69; percentage of folds 0.014 vs. 0.017; $r>0.95$ vs. $r<0.5$ |
| Deep ICP covariance estimation [2509.18954] | KITTI | Up to $\sim10\%$ reduction in Absolute Pose Error and several-percent gains in Relative Pose Error |
| X-ICP [2211.16335] | Straight corridors | Translation error $\approx 0.04$ m, rotation error $\approx 0.5^\circ$, success 95% |

Additional LiDAR evidence comes from LP-ICP, which achieves the best or comparable RMSE on all 10 simulated lunar-like sequences; on “a2_traverse” (2.231 km), LP-ICP reports RMSE $=12.31$ m versus $36.6$ m for the Zhang et al. degeneracy method and $58.4$ m for X-ICP, while remaining real-time capable at $\sim35$ ms per ICP on ANYmal 1 [2501.02580]. In U-ViLAR ablations, removing the pose distribution prior $\mathbf P_{\mathrm{uncert}}$ in registration raises the lateral/longitudinal MAE and especially the RMSE, indicating that the prior is not merely descriptive but materially sharpens the final pose [2507.04503].

## 5. Uncertainty, registration error, and calibration

A recurrent issue is whether registration uncertainty can be interpreted as a surrogate for registration error. Luo et al. examined this question directly for Gaussian-process nonrigid registration and found only a weak-to-moderate positive monotonic association between point-wise GP uncertainty and true non-rigid registration error: mean Spearman $\rho_s\simeq0.29$ on RESECT with manual landmarks and mean $\rho_s\simeq0.40$ on MIBS with automatic landmarks. Patch-wise appearance-based error measures, defined through SSD or histogram intersection, showed very low $\rho_s(\Omega)$ across patch sizes $k=3,5$. They therefore concluded that point-wise transformation uncertainty is statistically significant but not strong enough to guarantee that regions of low GP uncertainty always have low error, and that GP uncertainty can be misleading because it purely reflects distance to interpolation landmarks [1908.07709].

Subsequent work reinforces the need to distinguish useful uncertainty from poorly calibrated uncertainty. In hierarchical neuroimaging registration, aleatoric uncertainty learned as per-voxel variance achieved Spearman $\rho\approx0.60$ and Pearson $r\approx0.48$ with true coordinate-prediction error, whereas epistemic uncertainty via MC-dropout correlated poorly, with Spearman $\rho\approx0.18$ and Pearson $r\approx0.11$ [2410.09299]. PULPo similarly reported substantially better calibrated uncertainty quantification than probabilistic VoxelMorph; on OASIS-1, it obtained $\mathrm{NCC}_{VX}=0.533\pm0.039$ and $\mathrm{NCC}_{LM}=0.302\pm0.110$, and on BraTS-Reg it obtained $\mathrm{NCC}_{VX}=0.497\pm0.073$ and $\mathrm{NCC}_{LM}=0.229\pm0.324$. Its variance maps localized high uncertainty around resection cavities and lesion boundaries, whereas VoxelMorph’s maps were near-uniform and poorly calibrated [2407.10567].

NPBDREG addresses a related calibration problem from the perspective of out-of-distribution sensitivity. By sampling the posterior over network weights using SGLD-Adam, it reported a better correlation of predicted uncertainty with out-of-distribution data than probabilistic VoxelMorph, specifically $r>0.95$ versus $r<0.5$, while also improving registration accuracy and smoothness [2108.06771]. In U-ViLAR, the qualitative observation is narrower but consistent with this trend: frames with high predicted uncertainty $U_d$ exhibit larger residual errors, suggesting that the uncertainty estimate can flag difficult scenarios even when the module’s primary role is to guide a local differentiable search rather than to certify correctness [2507.04503].

The common lesson is not that uncertainty and error are interchangeable, but that uncertainty can become operationally useful when its representation is aligned with the underlying registration mechanism. Weak calibration yields weak guidance; calibrated uncertainty can support weighting, constrained optimization, posterior sampling, or downstream review.

## 6. Limitations and research directions

Several limitations recur across the literature. In heteroscedastic unsupervised registration, the noise model is Gaussian; a heteroscedastic Laplacian variant was also tested, but Gaussian performed best. The SNR weighting map
\[
w(x)=\mathcal T\bigl[(I/\hat\sigma)^{2\gamma}\bigr]
\]
is hand-crafted, and the authors state that more data-driven weighting might further boost performance. Collaborative training also requires a warmup stage and alternating steps, so hyperparameters such as $N_w$, $\gamma$, $\lambda$, and $\beta$ matter [2312.00836].

For transformation-uncertainty methods, the main conceptual limitation is that the estimated uncertainty need not measure the quantity of interest. Luo et al. explicitly recommend investigating joint modeling of transformation and label uncertainty, adaptive schemes that select new landmarks in regions of high uncertainty, alternative kernels or non-stationary GP priors that incorporate boundary or tissue-specific deformation models, and regional or task-driven correlation tests rather than whole-image analyses [1908.07709]. This suggests that uncertainty-guided registration is most reliable when the uncertainty model is tailored to the downstream notion of failure.

In localizability-aware point cloud registration, LP-ICP notes that choices of $(h_f,h_u,T_1\!-\!T_5,\mu_j)$ matter, and that observed variations in partially localizable directions suggest the need for further investigation on robustness and generalizability. Proposed extensions include propagating sensor covariance to a full $6\times6$ posterior, extending the method to multi-sensor fusion, and learning-based tuning of thresholds or weights [2501.02580]. Deep ICP covariance prediction identifies additional failure modes: highly dynamic scenes are not explicitly handled, very sparse point clouds may still yield overconfident estimates outside the training distribution, and the framework assumes a static environment and LiDAR-only input during covariance prediction. Suggested extensions include semantic filtering, attention-based backbones, multi-sensor EKF or factor-graph fusion, and simulated dynamic-object perturbations during training [2509.18954].

A broader implication is that localization uncertainty-guided registration is not a single algorithmic template but a class of registration strategies in which uncertainty influences search, weighting, fitting, constraint handling, or posterior propagation. The strongest results arise when uncertainty is represented in the same space in which registration decisions are made: a discrete 3-DoF pose grid in BEV localization, per-voxel variance in transformation fitting, eigen-directions of a Hessian in constrained ICP, or posterior covariance in Bayesian diffeomorphic registration.

Source: https://www.emergentmind.com/topics/localization-uncertainty-guided-registration