Localization Uncertainty-Guided Registration

• The paper introduces a method that integrates discrete and continuous uncertainty models to refine registration, reducing errors in pose estimation.
• It details how localized uncertainty, whether in pose, voxel correspondence, or deformation parameters, guides the optimization process across multiple domains.
• The work demonstrates that embedding uncertainty into the registration process enhances robustness by mitigating the adverse effects of ambiguous or noisy data.

Localization uncertainty-guided registration denotes registration procedures in which an estimated uncertainty over pose, displacement, correspondences, or deformation parameters actively shapes the alignment step rather than being reported only after optimization. In U-ViLAR, the term refers to a module that begins with a coarse camera pose prior $\mathbf P_0=(x_0,y_0,\varphi_0)$, models the remaining ambiguity as a 3-DoF probability distribution, and uses that distribution to guide a local, fine-grained, differentiable registration in Bird’s-Eye-View (BEV) space (Li et al., 6 Jul 2025). Closely related formulations appear in neuroimaging, where per-voxel Gaussian uncertainty is propagated to global transformation models and used for uncertainty-aware fitting, and in unsupervised medical image registration, where heteroscedastic image uncertainty reduces the influence of regions with high uncertainty during training (Hu et al., 2024, Zhang et al., 2023). In LiDAR and landmark-based registration, analogous mechanisms use local information matrices, predicted ICP error covariance, or Bayesian posteriors to constrain ill-conditioned directions, reweight observations, or quantify deformation variability (Tuna et al., 2022, Marsland et al., 2016).

1. Problem setting and sources of uncertainty

Registration is repeatedly described in the literature as an ambiguous estimation problem whose failure modes are strongly domain-dependent. In urban visual localization, even after perceptual-uncertainty-aware association, the coarse pose estimate $(x_c,y_c,\varphi_c)$ may remain uncertain or multi-modal because of symmetric street layouts or occlusion; the refinement stage must therefore search locally around an uncertain coarse hypothesis rather than trust a single pose estimate (Li et al., 6 Jul 2025). In unsupervised medical image registration, the standard homoscedastic assumption implicit in mean-squared-error objectives is often violated because medical images exhibit heteroscedastic and input-dependent noise; this can cause spurious gradients from noise-induced outliers and degrade displacement estimation (Zhang et al., 2023).

In geometric localization, uncertainty also arises from lack of observability rather than from measurement noise alone. X-ICP characterizes LiDAR-challenging environments as settings in which geometrically uninformative scans deteriorate point cloud registration performance and push optimization toward divergence along weakly constrained directions; the corresponding uncertainty is expressed as fine-grained localizability of individual pose degrees of freedom (Tuna et al., 2022). A related but distinct source appears in building-level LiDAR-to-model alignment, where L2M-Reg explicitly addresses the generalization uncertainty in semantic 3D city models at the Level of Detail 2 (LoD2) through reliable plane correspondence establishment, a pseudo-plane-constrained Gauss-Helmert model, and adaptive estimation of vertical translation (Xu et al., 20 Sep 2025).

Earlier work in landmark-based image registration emphasized that even small changes in landmark positions can produce large changes in the resulting diffeomorphism. Marsland and Shardlow therefore introduced a Langevin formulation for small random perturbations of Hamiltonian landmark registration, making localization noise a first-class variable in the prior and posterior over deformations (Marsland et al., 2016). Taken together, these formulations show that “localization uncertainty” can refer to uncertainty in a coarse global pose, in voxel-wise correspondence coordinates, in deformation parameters, or in the conditioning of the registration objective itself.

2. Mathematical representations of localization uncertainty

A central design choice is how the uncertainty is parameterized. In U-ViLAR, the registration stage is defined on a discrete grid of candidate poses

$$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$

and a pose decoder outputs three independent discrete distributions $\mathbf p_x\in\mathbb R^{N_x}$, $\mathbf p_y\in\mathbb R^{N_y}$, and $\mathbf p_\varphi\in\mathbb R^{N_\varphi}$. Their per-dimension entropies

$$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$

quantify uncertainty in each degree of freedom, while the joint prior over the 3-DoF grid is the product

$$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$

This representation is discrete, differentiable, and naturally tied to a local search volume (Li et al., 6 Jul 2025).

In Gaussian-process registration, Luo et al. represent transformation uncertainty by the posterior covariance of a nonrigid deformation field. For each scalar component $d(x)$ of the vector-valued deformation $u(x)$, the prior is $(x_c,y_c,\varphi_c)$0 with $(x_c,y_c,\varphi_c)$1 and $(x_c,y_c,\varphi_c)$2. Given observed landmark displacements, the GP posterior at new locations has mean $(x_c,y_c,\varphi_c)$3 and covariance $(x_c,y_c,\varphi_c)$4. Point-wise transformation uncertainty is then

$(x_c,y_c,\varphi_c)$5

and point-wise registration error is

$(x_c,y_c,\varphi_c)$6

Here uncertainty is attached directly to the predicted displacement field rather than to a local search distribution (Luo et al., 2019).

Learning-based neuroimaging work adopts a per-voxel Gaussian model. The network predicts a mean $(x_c,y_c,\varphi_c)$7 and standard deviation $(x_c,y_c,\varphi_c)$8 for each voxel $(x_c,y_c,\varphi_c)$9 and direction $$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$0,

$$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$1

leading to the Gaussian-negative-log-likelihood

$$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$2

These local variances are then lifted to global parameter uncertainty through weighted least squares and covariance propagation (Hu et al., 2024).

A path-space variant appears in the landmark framework of Marsland and Shardlow. Starting from Hamiltonian geodesic registration, they introduce a generalized Langevin SDE on momenta and landmark positions,

$$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$3

with $$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$4. The invariant Gibbs density is proportional to $$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$5, so localization uncertainty is represented as a thermally perturbed prior over diffeomorphic registrations (Marsland et al., 2016).

3. Mechanisms by which uncertainty guides registration

Once uncertainty is represented, the key methodological question is how it enters the registration objective. In U-ViLAR, each candidate pose $$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$6 produces a raw feature-difference cost

$$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$7

This cost is fused with the learned uncertainty prior as

$$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$8

then processed by a small 3D CNN to produce a final cost volume $$\Omega=\{\theta_{i,j,k}=(x_c-\Delta_x+i\,\delta_x,\;y_c-\Delta_y+j\,\delta_y,\;\varphi_c-\Delta_\varphi+k\,\delta_\varphi)\},$$9. The refined pose is obtained by a 3D softmax weighted average,

$\mathbf p_x\in\mathbb R^{N_x}$0

with learned scalars $\mathbf p_x\in\mathbb R^{N_x}$1 and $\mathbf p_x\in\mathbb R^{N_x}$2. The entire refinement is differentiable, and no external iterative solver or hard PnP step is used (Li et al., 6 Jul 2025).

In heteroscedastic medical image registration, uncertainty guidance takes the form of adaptive residual weighting. The fixed image is modeled under a zero-mean heteroscedastic Gaussian noise model,

$\mathbf p_x\in\mathbb R^{N_x}$3

which yields the heteroscedastic NLL

$\mathbf p_x\in\mathbb R^{N_x}$4

Because direct joint optimization of the displacement and variance estimators proved unstable, the framework uses a collaborative or alternating training schedule, with a warm-up stage followed by alternating displacement and variance steps. The displacement loss uses a relative SNR weight

$\mathbf p_x\in\mathbb R^{N_x}$5

so that high-noise, low-SNR regions are down-weighted. Blocking gradients on $\mathbf p_x\in\mathbb R^{N_x}$6 prevents the displacement network from inflating the variance estimate to suppress its own loss (Zhang et al., 2023).

In hierarchical neuroimaging registration, uncertainty guidance is expressed as uncertainty-aware fitting of a global transformation model. If $\mathbf p_x\in\mathbb R^{N_x}$7 and $\mathbf p_x\in\mathbb R^{N_x}$8, the weighted least-squares fit is

$\mathbf p_x\in\mathbb R^{N_x}$9

The optimization is equivalent to minimizing

$\mathbf p_y\in\mathbb R^{N_y}$0

Here uncertain voxels contribute less to global fitting, and the resulting coefficient covariance supports posterior sampling for downstream propagation (Hu et al., 2024).

In LiDAR registration, uncertainty guidance is often implemented as localizability-aware constraints. X-ICP computes the local information matrix

$\mathbf p_y\in\mathbb R^{N_y}$1

performs the eigenvalue decomposition $\mathbf p_y\in\mathbb R^{N_y}$2, and declares a principal direction $\mathbf p_y\in\mathbb R^{N_y}$3 degenerate when the normalized strength

$\mathbf p_y\in\mathbb R^{N_y}$4

falls below a threshold $\mathbf p_y\in\mathbb R^{N_y}$5. The constrained Gauss-Newton step enforces

$\mathbf p_y\in\mathbb R^{N_y}$6

or equivalently projects the update into the well-constrained subspace. LP-ICP generalizes this strategy by combining point-to-line and point-to-plane residuals, computing per-direction localizability contributions from Hessian blocks, and adding soft constraints for partially localizable directions together with hard constraints for non-localizable ones (Tuna et al., 2022, Yue et al., 5 Jan 2025).

A complementary mechanism is to predict the uncertainty of a registration module before or during state estimation. In deep ICP covariance estimation, a network predicts a $\mathbf p_y\in\mathbb R^{N_y}$7 SPD covariance $\mathbf p_y\in\mathbb R^{N_y}$8 for each LiDAR scan. In an EKF on $\mathbf p_y\in\mathbb R^{N_y}$9, the ICP residual is

$\mathbf p_\varphi\in\mathbb R^{N_\varphi}$0

the innovation covariance is

$\mathbf p_\varphi\in\mathbb R^{N_\varphi}$1

and the Kalman gain uses the predicted registration covariance directly. In this setting, uncertainty does not only annotate the ICP output; it changes the filter update that follows the registration (Dolatabadi et al., 23 Sep 2025).

At building level, L2M-Reg uses an uncertainty-aware plane-based fine registration pipeline consisting of reliable plane correspondence establishment, a pseudo-plane-constrained Gauss-Helmert model, and adaptive estimation of vertical translation, explicitly accounting for model uncertainty in semantic 3D city models (Xu et al., 20 Sep 2025).

4. Representative systems and reported performance

Reported benefits are heterogeneous because the underlying tasks differ, but the empirical pattern is that uncertainty enters as a useful control signal when it is tightly coupled to the registration mechanism rather than appended as a post hoc score. Representative results span BEV visual localization, medical image registration, brain MRI registration, and LiDAR localization (Li et al., 6 Jul 2025, Zhang et al., 2023, Hu et al., 2024, 2108.06771, Tuna et al., 2022, Dolatabadi et al., 23 Sep 2025).

System	Domain	Reported result
U-ViLAR (Li et al., 6 Jul 2025)	nuScenes + HD map	Including LU-Guided Registration reduces orientation MAE on nuScenes from ~0.116° to 0.075°
Heteroscedastic framework (Zhang et al., 2023)	ACDC	VoxelMorph/MSE: 80.20%; VoxelMorph + NLL: 76.49%; VoxelMorph + β-NLL: 78.74%; Ours (γ=0.5): 80.73%
Hierarchical uncertainty estimation (Hu et al., 2024)	ABIDE / OASIS3	Affine Dice 0.718→0.730 and 0.673→0.682; B-spline (10 mm) 0.782→0.790 and 0.750→0.772
NPBDREG (2108.06771)	Brain MRI	Dice score 0.74 vs. 0.69; percentage of folds 0.014 vs. 0.017; $\mathbf p_\varphi\in\mathbb R^{N_\varphi}$2 vs. $\mathbf p_\varphi\in\mathbb R^{N_\varphi}$3
Deep ICP covariance estimation (Dolatabadi et al., 23 Sep 2025)	KITTI	Up to $\mathbf p_\varphi\in\mathbb R^{N_\varphi}$4 reduction in Absolute Pose Error and several-percent gains in Relative Pose Error
X-ICP (Tuna et al., 2022)	Straight corridors	Translation error $\mathbf p_\varphi\in\mathbb R^{N_\varphi}$5 m, rotation error $\mathbf p_\varphi\in\mathbb R^{N_\varphi}$6, success 95%

Additional LiDAR evidence comes from LP-ICP, which achieves the best or comparable RMSE on all 10 simulated lunar-like sequences; on “a2_traverse” (2.231 km), LP-ICP reports RMSE $\mathbf p_\varphi\in\mathbb R^{N_\varphi}$7 m versus $\mathbf p_\varphi\in\mathbb R^{N_\varphi}$8 m for the Zhang et al. degeneracy method and $\mathbf p_\varphi\in\mathbb R^{N_\varphi}$9 m for X-ICP, while remaining real-time capable at $$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$0 ms per ICP on ANYmal 1 (Yue et al., 5 Jan 2025). In U-ViLAR ablations, removing the pose distribution prior $$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$1 in registration raises the lateral/longitudinal MAE and especially the RMSE, indicating that the prior is not merely descriptive but materially sharpens the final pose (Li et al., 6 Jul 2025).

5. Uncertainty, registration error, and calibration

A recurrent issue is whether registration uncertainty can be interpreted as a surrogate for registration error. Luo et al. examined this question directly for Gaussian-process nonrigid registration and found only a weak-to-moderate positive monotonic association between point-wise GP uncertainty and true non-rigid registration error: mean Spearman $$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$2 on RESECT with manual landmarks and mean $$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$3 on MIBS with automatic landmarks. Patch-wise appearance-based error measures, defined through SSD or histogram intersection, showed very low $$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$4 across patch sizes $$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$5. They therefore concluded that point-wise transformation uncertainty is statistically significant but not strong enough to guarantee that regions of low GP uncertainty always have low error, and that GP uncertainty can be misleading because it purely reflects distance to interpolation landmarks (Luo et al., 2019).

Subsequent work reinforces the need to distinguish useful uncertainty from poorly calibrated uncertainty. In hierarchical neuroimaging registration, aleatoric uncertainty learned as per-voxel variance achieved Spearman $$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$6 and Pearson $$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$7 with true coordinate-prediction error, whereas epistemic uncertainty via MC-dropout correlated poorly, with Spearman $$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$8 and Pearson $$U_d=-\sum_{n=1}^{N_d} p_d^{(n)}\log p_d^{(n)},\quad d\in\{x,y,\varphi\},$$9 (Hu et al., 2024). PULPo similarly reported substantially better calibrated uncertainty quantification than probabilistic VoxelMorph; on OASIS-1, it obtained $$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$0 and $$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$1, and on BraTS-Reg it obtained $$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$2 and $$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$3. Its variance maps localized high uncertainty around resection cavities and lesion boundaries, whereas VoxelMorph’s maps were near-uniform and poorly calibrated (Siegert et al., 2024).

NPBDREG addresses a related calibration problem from the perspective of out-of-distribution sensitivity. By sampling the posterior over network weights using SGLD-Adam, it reported a better correlation of predicted uncertainty with out-of-distribution data than probabilistic VoxelMorph, specifically $$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$4 versus $$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$5, while also improving registration accuracy and smoothness (2108.06771). In U-ViLAR, the qualitative observation is narrower but consistent with this trend: frames with high predicted uncertainty $$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$6 exhibit larger residual errors, suggesting that the uncertainty estimate can flag difficult scenarios even when the module’s primary role is to guide a local differentiable search rather than to certify correctness (Li et al., 6 Jul 2025).

The common lesson is not that uncertainty and error are interchangeable, but that uncertainty can become operationally useful when its representation is aligned with the underlying registration mechanism. Weak calibration yields weak guidance; calibrated uncertainty can support weighting, constrained optimization, posterior sampling, or downstream review.

6. Limitations and research directions

Several limitations recur across the literature. In heteroscedastic unsupervised registration, the noise model is Gaussian; a heteroscedastic Laplacian variant was also tested, but Gaussian performed best. The SNR weighting map

$$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$7

is hand-crafted, and the authors state that more data-driven weighting might further boost performance. Collaborative training also requires a warmup stage and alternating steps, so hyperparameters such as $$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$8, $$\mathbf P_{\mathrm{uncert}(i,j,k)}=\mathbf p_x^{(i)}\;\mathbf p_y^{(j)}\;\mathbf p_\varphi^{(k)}.$$9, $d(x)$0, and $d(x)$1 matter (Zhang et al., 2023).

For transformation-uncertainty methods, the main conceptual limitation is that the estimated uncertainty need not measure the quantity of interest. Luo et al. explicitly recommend investigating joint modeling of transformation and label uncertainty, adaptive schemes that select new landmarks in regions of high uncertainty, alternative kernels or non-stationary GP priors that incorporate boundary or tissue-specific deformation models, and regional or task-driven correlation tests rather than whole-image analyses (Luo et al., 2019). This suggests that uncertainty-guided registration is most reliable when the uncertainty model is tailored to the downstream notion of failure.

In localizability-aware point cloud registration, LP-ICP notes that choices of $d(x)$2 matter, and that observed variations in partially localizable directions suggest the need for further investigation on robustness and generalizability. Proposed extensions include propagating sensor covariance to a full $d(x)$3 posterior, extending the method to multi-sensor fusion, and learning-based tuning of thresholds or weights (Yue et al., 5 Jan 2025). Deep ICP covariance prediction identifies additional failure modes: highly dynamic scenes are not explicitly handled, very sparse point clouds may still yield overconfident estimates outside the training distribution, and the framework assumes a static environment and LiDAR-only input during covariance prediction. Suggested extensions include semantic filtering, attention-based backbones, multi-sensor EKF or factor-graph fusion, and simulated dynamic-object perturbations during training (Dolatabadi et al., 23 Sep 2025).

A broader implication is that localization uncertainty-guided registration is not a single algorithmic template but a class of registration strategies in which uncertainty influences search, weighting, fitting, constraint handling, or posterior propagation. The strongest results arise when uncertainty is represented in the same space in which registration decisions are made: a discrete 3-DoF pose grid in BEV localization, per-voxel variance in transformation fitting, eigen-directions of a Hessian in constrained ICP, or posterior covariance in Bayesian diffeomorphic registration.