Perceptual Uncertainty-Guided Association

• Perceptual Uncertainty-guided Association is a design principle that uses explicit uncertainty estimates to determine the strength and reliability of linking sensor observations to representations.
• It is applied in diverse domains such as collaborative perception, visual localization, 3D multi-object tracking, belief space planning, and neuro-symbolic reasoning.
• The approach integrates uncertainty directly into association operators through techniques like soft correspondence refinement, modified divergence measures, and uncertainty-weighted fusion to enhance robustness and accuracy.

Searching arXiv for the papers most directly connected to perceptual uncertainty-guided association. Search 1: direct visual-localization formulation of “Perceptual Uncertainty-guided Association”. Search 2: collaborative-perception formulation using physically grounded uncertainty-guided spatial association. Search 3: uncertainty-guided data association in 3D multi-object tracking. Perceptual uncertainty-guided association denotes a class of association mechanisms in which correspondence, weighting, or commitment is controlled by an explicit estimate of perceptual reliability rather than by a detector score or a deterministic state alone. In the literature, this idea appears in several technically distinct forms: BEV-cell-wise weighting for collaborative perception, visual-to-map correspondence in visual localization, distribution-based track–detection matching in 3D multi-object tracking, mixture-aware scene association in belief space planning, and probabilistic predicate selection in neuro-symbolic reasoning (Yang et al., 22 Jun 2026, Li et al., 6 Jul 2025, He et al., 2023, Pathak et al., 2016, Wu et al., 18 Nov 2025). The unifying principle is that uncertainty is treated as a first-class signal that shapes association itself, rather than as a post-hoc diagnostic.

1. Definition and conceptual scope

At its narrowest, perceptual uncertainty-guided association means using uncertainty on the perception side to decide how strongly an observation should be linked to another representation. The relevant representation may be another agent’s BEV feature map, a map cell, a predicted track, a scene hypothesis, or a symbolic predicate. The uncertainty signal may be a spatial reliability field, a covariance matrix, a probability distribution over associations, or a confidence-calibrated symbolic state.

The concept is especially clear in two formulations that use the term directly. In U-ViLAR, “Perceptual Uncertainty-guided Association” is a BEV-space visual–map matching module that predicts a pixel-wise visual uncertainty field and injects it into a global similarity matrix before coarse pose decoding (Li et al., 6 Jul 2025). In UECP, the same functional idea appears as “use a physically grounded uncertainty signal, rather than a detector-derived confidence signal, to control the contribution of each agent’s features during fusion,” so that association is realized as uncertainty-weighted aggregation over aligned BEV cells rather than as explicit object-level matching (Yang et al., 22 Jun 2026).

A common misconception is that association in this setting must mean discrete one-to-one assignment. That is only one case. In UECP, association is implicit and spatial: after ego-frame transformation and visibility masking, uncertainty determines how much each aligned BEV cell from each agent contributes to the fused ego representation (Yang et al., 22 Jun 2026). In UG3DMOT and earlier fuzzy-belief approaches, by contrast, association is explicitly pairwise and one-to-one, with a final assignment stage over candidate correspondences (He et al., 2023, Gruyer et al., 2013). This suggests that perceptual uncertainty-guided association is better understood as a design principle than as a single algorithmic family.

2. Association primitives and mathematical forms

A recurring structural pattern is to insert uncertainty directly into the association operator. In UECP, collaborative perception over a communication graph $G$ is formulated as

$$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$

with collaborator messages

$$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$

Here uncertainty $\mathcal{U}_i$ is not auxiliary metadata; it is an explicit input to communication and fusion (Yang et al., 22 Jun 2026).

In U-ViLAR, the primitive is a differentiable soft correspondence distribution between visual BEV units and map BEV units. A raw similarity matrix $$\mathcal S(i,j)=\langle \mathbf F_v^{\mathrm{BEV}(i)}, \mathbf F_m^{\mathrm{BEV}(j)}\rangle$$ is concatenated with tiled perceptual uncertainty and refined as

$$\mathcal S_{\text{uncert}} = \mathrm{CNN}\big([\mathcal S,\mathbf U_P^{\text{tile}}]\big),$$

followed by row-wise softmax

$$\mathbf P_{i,j}= \frac{\exp(\mathcal S_{\text{uncert}(i,j)})}{\sum_k\exp(\mathcal S_{\text{uncert}(i,k)})}.$$

Association is therefore a learned uncertainty-conditioned remapping of affinity logits rather than a hard nearest-neighbor rule (Li et al., 6 Jul 2025).

In UG3DMOT, tracks and detections are modeled as Gaussian random vectors, and association uses a modified divergence

$$D_{\mathrm{mod}}(D_i^t\|\hat{T}_j^t) = D_{\mathrm{JS}}(D_i^t\|\hat{T}_j^t)\,\alpha(D_i^t,\hat{T}_j^t),$$

with

$$\alpha(D_i^t,\hat{T}_j^t)=2-\cos(\theta_{D_i},\theta_{\hat{T}_j}) \in[1,2],$$

and final uncertainty-guided cost

$$D_{\mathrm{res}}(D_i^t\|\hat{T}_j^t) = D_{\mathrm{mod}}(D_i^t\|\hat{T}_j^t)\,\operatorname{mean}(\hat{\Sigma}_j^t).$$

The distinctive point is that uncertainty first makes association more permissive through distributional similarity, then less permissive through explicit covariance weighting, so that highly uncertain tracks do not match too broadly (He et al., 2023).

In DA-BSP, association is elevated to a latent variable in belief evolution. The observation likelihood becomes

$$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$0

where each $$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$1 integrates measurement likelihood, event likelihood, and predicted pose belief. The posterior consequently becomes a mixture over scene-association and prior-state hypotheses, and action selection minimizes expected cost over that mixture rather than over a single associated posterior (Pathak et al., 2016). This suggests a broader operational definition: perceptual uncertainty-guided association is any mechanism in which uncertainty changes the feasible correspondence set, the relative weight of candidate correspondences, or the willingness to commit to a single correspondence.

3. Spatial association in BEV: collaborative perception and visual localization

UECP provides the clearest physically grounded formulation. The paper argues that confidence maps derived from the classification head are inherently entangled with the detector’s own errors and biases, producing a self-reinforcement problem in which a false positive with high classification confidence receives large fusion weight and becomes harder to suppress. Its alternative is an uncertainty map supervised directly from LiDAR point density, using BEV rasterization at $$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$2 and target

$$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$3

so that sparse cells have high uncertainty and dense cells low uncertainty (Yang et al., 22 Jun 2026).

This uncertainty enters fusion at two levels. Uncertainty-Weighted Downsampling preserves reliable information when building the pyramid,

$$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$4

and Uncertainty-Guided Residual Fusion computes per-agent, per-cell weights

$$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$5

before residual correction around the ego feature. The resulting association is soft, spatial, multi-agent, and multi-scale. Empirically, UECP reports $$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$6 AP@30/50/70 on DAIR-V2X and $$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$7 on V2V4REAL, with gains strongest at AP@70; it also adds only one uncertainty channel to a $$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$8-channel BEV feature tensor, corresponding to approximately $$\max_{\theta} \quad g\left(\Phi_\theta\left(\mathcal{X}_i,\mathcal{U}_i, \{\mathcal{M}_{j\to i}\}_{j\in\mathcal{N}(i)} \right),\mathcal{O}_i\right),$$9 additional bandwidth (Yang et al., 22 Jun 2026).

U-ViLAR applies the same principle to visual localization, but with a learned heteroscedastic visual uncertainty field rather than LiDAR-density supervision. Perceptual uncertainty is modeled as $$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$0 and learned via

$$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$1

so high-uncertainty regions have relaxed alignment constraints while confident regions are forced to align more strictly. Global association is trained against a Gaussian soft target

$$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$2

with cross-entropy loss on the uncertainty-aware assignment matrix $$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$3. The paper reports that removing Perceptual Uncertainty and using the original Similarity Matrix causes a significant increase in localization error, whereas Local Association has a relatively small impact on localization accuracy; it also reports 28 FPS on NVIDIA V100 and 15 FPS on NVIDIA Orin with TensorRT INT8 quantization on the BEV encoder (Li et al., 6 Jul 2025).

Taken together, these two systems establish a specific subgenre of the topic: spatial association in BEV guided by a reliability field. Their major point of agreement is that uncertainty should modulate correspondence before downstream estimation, not after it.

4. Object-level association, ambiguity removal, and belief evolution

In 3D MOT, perceptual uncertainty-guided association is instantiated at the level of track–detection correspondence. UG3DMOT represents each detection as a 7-dimensional Gaussian box state with diagonal covariance and each track as a 10-dimensional Gaussian Kalman state under a constant velocity model. The paper’s central claim is that deterministic tracks and detections are inadequate because both track prediction and detection are inherently uncertain. Using Jensen–Shannon divergence between probabilistic states, plus heading penalty and covariance-guided cost scaling, the method reports the best listed HOTA and the lowest IDSW on KITTI and the best overall AMOTA on nuScenes; ablations further state that uncertainty-guided design reduces FN, FP, and IDSW relative to a version with no uncertainty involved (He et al., 2023).

An older but conceptually related approach is the belief-theoretic multi-object association method for intelligent vehicles. There, perceived objects are represented by fuzzy measures and known objects by fuzzy prediction windows; support represents inaccuracy and height represents uncertainty. Pairwise concordance is mapped into belief masses over association, non-association, ignorance, and reject, then global ambiguity is removed with the Hungarian algorithm. The final confidence of the selected assignment set is

$$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$4

This formulation is notable because it explicitly includes “nothing” hypotheses for appearance, disappearance, and false alarms, so uncertainty may lead to deliberate non-association rather than forced matching (Gruyer et al., 2013).

DA-BSP generalizes the same idea from tracking to planning. It rejects the standard BSP assumption that data association is perfect and shows that, under perceptual aliasing and localization uncertainty, the posterior becomes a mixture over scene hypotheses and prior belief modes. The ambiguity term

$$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$5

penalizes actions that leave association weights close to uniform, while the overall planning objective evaluates future actions under association-aware likelihoods and mixture posteriors. In this setting, perceptual uncertainty-guided association does not merely improve inference; it drives action selection toward discriminative viewpoints and active disambiguation (Pathak et al., 2016).

These three lines of work clarify a second misconception: uncertainty-guided association is not synonymous with softer pairwise costs. In tracking it can regulate affinity, in belief-theoretic matching it can license reject states, and in planning it can change which sensing action is chosen next.

5. Symbolic, relational, and active forms of association

In neuro-symbolic reasoning, the same principle appears after the perception stack rather than inside a geometric matcher. The translator $$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$6 maps observations to a probabilistic symbolic state

$$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$7

and predicate-level uncertainty is defined as

$$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$8

The paper then refines state uncertainty with a dependency-aware MRF,

$$\mathcal{M}_{j\to i} = \mathcal{P}(\mathcal{X}_j, \mathcal{U}_j).$$9

and uses thresholding to classify predicates as Certain True, Certain False, or Uncertain before planning. If critical predicates remain uncertain, the planner executes information-gathering actions such as $\mathcal{U}_i$0 or $\mathcal{U}_i$1, justified by the condition

$\mathcal{U}_i$2

Functionally, uncertain object–relation links are associated to planning facts only when confidence is adequate; otherwise association is postponed and new evidence is sought. The system reports $\mathcal{U}_i$3, $\mathcal{U}_i$4, and $\mathcal{U}_i$5 success on Simple Stack, Deep Stack, and Clear+Stack, and drops from $\mathcal{U}_i$6 to $\mathcal{U}_i$7 on YCB-Video Complex Stack when information gathering is removed (Wu et al., 18 Nov 2025).

A behavioral analogue appears in the study of human drivers from the perspective of perceptual uncertainty reduction. There, uncertainty is not an explicit association score but is inferred from changes in SHAP saliency distributions over a merge maneuver. KL divergence between adjacent saliency distributions decreases over the decision process, while mutual information increases, and the most salient features narrow toward ego speed and relative lead-vehicle motion. This suggests, as an interpretation rather than a direct algorithmic claim, that trustworthy sequential decisions are preceded by progressive concentration on a smaller set of uncertainty-reducing cues (Wang et al., 2022).

These symbolic and behavioral formulations broaden the topic beyond vision-only matching. They show that perceptual uncertainty can guide not only which correspondence is selected, but also whether a symbolic commitment should be made at all, and whether more information should be acquired before committing.

6. Measurement, calibration, and limitations

A general framework for the topic requires a broader notion of perceptual uncertainty than detector confidence. The safety-oriented framework for automated driving defines perceptual uncertainty as the uncertain or limited knowledge of the true state of the world produced by perception and identifies seven influence factors: conceptual uncertainty, development situation and scenario coverage, situation or scenario uncertainty, sensor properties, labeling uncertainty, model uncertainty, and operational domain uncertainty. In that view, model confidence is only one influence factor among several, and runtime assessment should include scene conditions, sensor behavior, confidence measures, and novelty detection (Czarnecki et al., 2019).

Calibration and uncertainty decomposition become especially important when association behavior depends directly on uncertainty values. A recent feature-space method argues that most systems collapse all uncertainty modes into a single confidence score and instead decomposes uncertainty into aleatoric

$\mathcal{U}_i$8

and epistemic

$\mathcal{U}_i$9

components computed from deep features without sampling, ensembling, or additional forward passes. Although the paper is not itself an association paper, it reports empirical orthogonality with mean pairwise $$\mathcal S(i,j)=\langle \mathbf F_v^{\mathrm{BEV}(i)}, \mathbf F_m^{\mathrm{BEV}(j)}\rangle$$0, MOT17 aleatoric–epistemic correlation $$\mathcal S(i,j)=\langle \mathbf F_v^{\mathrm{BEV}(i)}, \mathbf F_m^{\mathrm{BEV}(j)}\rangle$$1, about $$\mathcal S(i,j)=\langle \mathbf F_v^{\mathrm{BEV}(i)}, \mathbf F_m^{\mathrm{BEV}(j)}\rangle$$2 narrower prediction intervals at matched coverage around $$\mathcal S(i,j)=\langle \mathbf F_v^{\mathrm{BEV}(i)}, \mathbf F_m^{\mathrm{BEV}(j)}\rangle$$3, and $$\mathcal S(i,j)=\langle \mathbf F_v^{\mathrm{BEV}(i)}, \mathbf F_m^{\mathrm{BEV}(j)}\rangle$$4 average computational savings versus $$\mathcal S(i,j)=\langle \mathbf F_v^{\mathrm{BEV}(i)}, \mathbf F_m^{\mathrm{BEV}(j)}\rangle$$5 for a total-uncertainty baseline. This suggests that future association systems may benefit from distinguishing visually corrupted observations from unsupported or out-of-distribution embeddings, rather than collapsing both into one trust value (Kumar et al., 15 Nov 2025).

The existing literature also exhibits clear assumptions and limitations. UECP assumes that features and uncertainty maps can be transformed into a common ego frame with sufficient geometric accuracy, and its physical grounding is strongest in LiDAR-based or LiDAR-dominant settings; its uncertainty branch is decoupled from detection noise in a supervisory and semantic sense rather than through absolute architectural independence (Yang et al., 22 Jun 2026). U-ViLAR does not specify the exact uncertainty-head architecture or the exact global multi-task loss weighting, even though the association losses are explicit (Li et al., 6 Jul 2025). UG3DMOT does not fully specify how multidimensional Gaussian JS divergence is implemented in practice when the midpoint distribution becomes a mixture, and its final uncertainty guidance compresses track covariance to a scalar mean covariance (He et al., 2023). The neuro-symbolic framework is validated on 3–10 object tabletop scenes, depends heavily on calibration quality, and concentrates residual failure in $$\mathcal S(i,j)=\langle \mathbf F_v^{\mathrm{BEV}(i)}, \mathbf F_m^{\mathrm{BEV}(j)}\rangle$$6 detection, whose F1 is $$\mathcal S(i,j)=\langle \mathbf F_v^{\mathrm{BEV}(i)}, \mathbf F_m^{\mathrm{BEV}(j)}\rangle$$7 (Wu et al., 18 Nov 2025).

Across these formulations, the central technical lesson is consistent. Perceptual uncertainty-guided association is most distinctive when uncertainty is neither a generic confidence surrogate nor a post-hoc score, but an explicit control variable that decides how evidence is weighted, when correspondence should remain ambiguous, and when more information must be gathered before commitment.