AI-Induced Perception Uncertainties

• AI-induced perception uncertainties are the indeterminacies in sensory and model outputs, clearly distinguishing between irreducible aleatoric noise and epistemic knowledge gaps.
• Recent methodologies integrate Bayesian inference, evidential learning, and calibration techniques to accurately decompose and quantify uncertainty in perception systems.
• Practical applications in autonomous driving, closed-loop control, and synthetic media highlight how managing these uncertainties improves safety, robustness, and human trust.

AI-induced perception uncertainties are the uncertainties attached to what an AI system perceives, infers, or reports about the world from sensory, symbolic, or human-provided evidence. In safety-oriented formulations, perceptual uncertainty is the dispersion over possible real-world states that could reasonably be attributed to the quantity being perceived, given the overall perception setup (Czarnecki et al., 2019). Across recent work, this uncertainty is usually decomposed into aleatoric uncertainty, which is tied to noise, ambiguity, occlusion, and intrinsic variability, and epistemic uncertainty, which is tied to limited knowledge, inadequate parameterization, data scarcity, or operational mismatch; together these quantities determine whether perception outputs can be trusted in downstream planning, control, fusion, explanation, and human oversight (Braun et al., 2023).

1. Conceptual scope and taxonomy

The core distinction in the literature is between uncertainty in the world-facing evidence and uncertainty in the system’s own stance. Aleatoric uncertainty captures irreducible variability in observations, including sensor noise, occlusion, ambiguous shading, weather effects, or annotation ambiguity. Epistemic uncertainty captures ignorance arising from limited training coverage, poor model support, biased datasets, rare events, or out-of-distribution conditions. In autonomous driving, this distinction is directly safety-relevant because high epistemic uncertainty flags unfamiliar situations, whereas high aleatoric uncertainty flags noisy or ambiguous sensor evidence (Braun et al., 2023).

A more general theoretical treatment argues that total uncertainty should not always be reduced to a simple independent sum. The classical identity $TU = EU + AU$ is explicitly generalized as $TU := \alpha_1 AU + \alpha_2 EU$, with $\alpha_1 + \alpha_2 > 1$, to accommodate dependence between data-driven randomness and knowledge deficits. The same work places interval models, credal sets, random sets, probability boxes, and an $\epsilon$-contamination neural architecture within a unified imprecise-probability framework for AI uncertainty (Shariatmadar, 2024).

Some work further distinguishes epistemic uncertainty in information from subjective uncertainty as the system’s own attitude of being uncertain. On this view, uncertainty is not exhausted by low-confidence outputs; it can also be realized as an interrogative attitude whose content is a question rather than a proposition, as in the symbolic form $?D(a)$. This expands the topic from uncertainty quantification to uncertainty realization, especially in systems that defer, query, or hedge rather than merely score alternatives (Rosa, 2 Mar 2026).

2. Mathematical representations and calibration

The dominant formalism in perception models is Bayesian or approximately Bayesian. A canonical predictive distribution is

$$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$

with uncertainty extracted from stochastic predictions. In radar-centric semantic segmentation, epistemic uncertainty is modeled by a factorized Gaussian posterior

$$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$

trained with the ELBO-style objective

$$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$

Predictive entropy

$$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$

and mutual information

$$MI(y,w \mid x,D)=H[p(y \mid x,D)]-E_{p(w\mid D)}[H[p(y \mid x,w)]]$$

then separate aleatoric and epistemic components at the cell level (Braun et al., 2023).

A different sampling-free construction is evidential learning. In uncertainty-aware panoptic localization and mapping, the semantic head predicts Dirichlet evidence $TU := \alpha_1 AU + \alpha_2 EU$0, with $TU := \alpha_1 AU + \alpha_2 EU$1, predictive mean $TU := \alpha_1 AU + \alpha_2 EU$2, and epistemic proxy $TU := \alpha_1 AU + \alpha_2 EU$3. This formulation keeps uncertainty in the conjugate Dirichlet domain and supports downstream evidence fusion in a BEV map without Monte Carlo sampling (Sirohi et al., 2024).

Calibration enters when probabilities or parametric variances must match empirical correctness. In embodied object search, temperature scaling rescales logits as

$TU := \alpha_1 AU + \alpha_2 EU$4

and per-pixel uncertainty is computed from normalized entropy

$TU := \alpha_1 AU + \alpha_2 EU$5

In transformer-based 3D detection, post-hoc Temperature Scaling and Platt Scaling are used for classification, while per-parameter temperatures recalibrate predicted Gaussian variances and von-Mises concentrations for box center and yaw; D-ECE and MCA are then used as calibration diagnostics (Prasanna et al., 2024, Beemelmanns et al., 15 May 2026).

3. Spatial perception, mapping, and sequential aggregation

A representative environment-perception system segments automotive radar scans into a $TU := \alpha_1 AU + \alpha_2 EU$6 grid centered at the host vehicle, with a cell size of $TU := \alpha_1 AU + \alpha_2 EU$7, and performs cell-wise classification into free, occupied, moving object, and unknown. Input features include number of detections, average Doppler velocity, average RCS, and relative time of recording. On 21,776 training scenes and 9,294 test scenes, the deterministic ASPP model reached mIoU $TU := \alpha_1 AU + \alpha_2 EU$8, the fully probabilistic Gaussian-weight model reached mIoU $TU := \alpha_1 AU + \alpha_2 EU$9, and the hybrid deterministic–probabilistic model preserved mIoU $\alpha_1 + \alpha_2 > 1$0 with near-baseline cost; the MC Dropout baselines reached $\alpha_1 + \alpha_2 > 1$1 and $\alpha_1 + \alpha_2 > 1$2. The same study showed that both epistemic and aleatoric uncertainties are strictly monotonically correlated with prediction precision via quantile-based analysis (Braun et al., 2023).

Sequential embodiment makes overconfidence especially costly. In object search, the average gap from ground-truth to learned perception is 25.8 percentage points, and policies trained under ground-truth semantics fail when deployed with noisy perception. Calibration and uncertainty-aware aggregation narrow this gap substantially: with EMSANet, temperature scaling reduced ECE from $\alpha_1 + \alpha_2 > 1$3 to $\alpha_1 + \alpha_2 > 1$4 and improved uECE by $\alpha_1 + \alpha_2 > 1$5 absolute; with the same policy, calibrated weighted aggregation raised RL success rate from $\alpha_1 + \alpha_2 > 1$6 to $\alpha_1 + \alpha_2 > 1$7, raised SPL from $\alpha_1 + \alpha_2 > 1$8 to $\alpha_1 + \alpha_2 > 1$9, and reduced FPR from $\epsilon$0 to $\epsilon$1 (Prasanna et al., 2024).

Map-based localization exhibits the same pattern. In uPLAM, per-pixel evidential uncertainties from a panoptic CNN are fused into a $\epsilon$2 BEV map, and map cells carry both semantic probabilities and uncertainty. Evidential fusion reached mapping mIoU $\epsilon$3 with uECE $\epsilon$4, compared with $\epsilon$5 and uECE $\epsilon$6 for the log-odds-softmax baseline. In localization, adding uncertainty-weighted intersections and landmark mIoU reduced translational MAE to $\epsilon$7 and RMSE to $\epsilon$8 (Sirohi et al., 2024). These results establish a general pattern: uncertainty estimates matter not only at the perception head, but also in temporal aggregation and map alignment.

4. Closed-loop control under imperfect perception

When perception feeds control directly, uncertainty becomes part of the control law rather than an auxiliary diagnostic. In adaptive cruise control from RGB input, an ensemble of $\epsilon$9 heteroscedastic regressors estimates distance headway with predictive mean

$?D(a)$0

and predictive variance

$?D(a)$1

which decomposes into aleatoric and epistemic terms. The controller then enforces chance constraints

$?D(a)$2

converted under Gaussian assumptions into deterministic safety margins that expand with $?D(a)$3. In OOD scenarios such as a firetruck in HardRainSunset, ensemble variance increased and the controller became more conservative; across 56 High-D trajectories, Time-to-Collision was mostly at least $?D(a)$4 and Time-to-Safety was typically at most $?D(a)$5 (Li et al., 2024).

A complementary control-theoretic program replaces opaque perception modules with explicit perception error models. One formulation models misdetection with continuous-time Markov chains and measurement noise with Wiener processes, producing a Markov-jump SDE whose mean-square stability can be certified by LMIs. Another discrete-time formulation adds bounded disturbance to capture low-frequency bias, yielding stochastic guaranteed-cost and stochastic optimal guaranteed-cost control synthesis via convex optimization (Yan et al., 15 Sep 2025, Yan et al., 15 Sep 2025).

These abstractions shift the design problem from “understanding the network internally” to “characterizing the perception errors it induces.” In the continuous-time ACC case, only the performance-guaranteed controller stabilized the system under high misdetection. In the discrete-time car-following case, the SOGCC controller achieved fast convergence and smooth actions, whereas IDM failed to converge under adverse perception conditions (Yan et al., 15 Sep 2025, Yan et al., 15 Sep 2025). The broader implication is that uncertainty-aware autonomy is not merely about estimating $?D(a)$6; it is about embedding uncertainty into constraints, Lyapunov arguments, and robustness margins.

5. Human-in-the-loop, agentic, and subjective uncertainty

Perception uncertainty is not restricted to sensors and model weights. In concept-based AI, humans intervene on intermediate concepts, and those interventions are themselves uncertain. A Concept Bottleneck Model uses $?D(a)$7, while test-time interventions replace predicted concepts with human-provided values $?D(a)$8. When models are trained on hard labels but receive uncertain interventions at test time, intervention efficacy collapses quickly: in UMNIST, CBMs and CEMs both degrade under $?D(a)$9–$$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$0, whereas training with uncertain concept labels improves robustness. The same work introduced UMNIST, CUB-S, and the UElic platform; CUB-S contains 4,984 dense, fine-grained concept-group annotations from 89 Prolific participants over 28 concept groups per image (Collins et al., 2023).

Uncertainty presentation also changes human reliance. In long-term Alzheimer’s prognosis, binary uncertainty encodings increased momentary confidence, while continuous encodings improved perceived reliability and helped users recognize model limitations. In the general-participant study, reliability was higher for continuous encoding ($$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$1 vs $$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$2), but confidence was higher for binary encoding ($$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$3 vs $$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$4). In the expert study, continuous encoding produced higher ratings on trustworthiness, reliability, technical competence, and personal attachment, while binary encoding again produced higher confidence (Reyes et al., 1 Feb 2026). This dissociation counters the common assumption that sharper-looking outputs automatically support calibrated trust.

Agentic systems introduce a further layer in which uncertainty becomes a control signal. In Dual-Process Agentic UQ, verbalized confidence $$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$5 and explanation $$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$6 are propagated through Uncertainty-Aware Memory and used to trigger Uncertainty-Aware Reflection when $$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$7, with an empirically validated plateau around $$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$8. The framework improved ALFWorld success rate from $$p(y \mid x, D) = \int p(y \mid x, w) p(w \mid D)\,dw \approx \frac{1}{N}\sum_{n=1}^{N} p(y \mid x, w_n),$$9 to $$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$0 relative to ReAct and WebShop success rate from $$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$1 to $$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$2, while improving trajectory-level calibration (Zhang et al., 22 Jan 2026). A related functional analysis argues that uncertainty can be realized as an interrogative attitude with question content rather than merely as a low-confidence proposition; this suggests that systems can be designed to search, defer, or query when perception fails to settle an issue (Rosa, 2 Mar 2026).

6. Misperception, illusions, and synthetic media

AI-induced perception uncertainty also appears when models and humans disagree about what counts as a valid visual scene. Illusion studies use color, brightness, geometric, depth, and motion illusions as controlled probes of contextual inference. Some CNNs can reproduce human brightness illusions under targeted supervision, and color constancy models can show human-like lightness stability, but alignment remains partial. Visual-LLMs show low overall alignment with human perception, and in “illusion of illusion” experiments newer systems still reduce misclassification only to around $$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$3 false positives; they also remain vulnerable to linguistic priming, with prompts such as “In this illusion…” raising false-positive rates (Yang et al., 17 Aug 2025).

Synthetic-media detection exposes a parallel form of uncertainty. In a benchmark of human and model perception of AI-generated images, humans showed a misclassification rate of $$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$4, whereas the top-performing model under the same setting showed a $$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$5 failure rate. Human difficulty was strongly category-dependent: Multiperson images reached $$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$6 accuracy, whereas Object images reached only $$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$7 (Lu et al., 2023). This is not purely a human limitation: detector performance degrades sharply under generator shift, sampling-method shift, and resolution shift.

A training-free uncertainty detector, WePe, makes this explicit by treating natural images as in-distribution and AI-generated images as OOD for large natural-image-pretrained encoders. Using DINOv2 ViT-L/14 and weight perturbation, it defines high uncertainty as instability of normalized features under small perturbations. On ImageNet-based benchmarks it achieved average AUROC $$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$8 and AP $$q(w \mid \theta) = \prod_j \mathcal{N}(w_j \mid \mu_j, \sigma_j^2),$$9; on LSUN-Bedroom it achieved average AUROC $$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$0 and AP $$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$1; on GenImage it achieved average accuracy $$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$2. It also remained comparatively robust under JPEG compression, Gaussian blur, and added Gaussian noise (Nie et al., 2024). The common thread across these studies is that perception uncertainty is not only a confidence deficit but also a measurable signature of misalignment between learned priors and actual scene structure.

7. Assurance, data quality, and deployment

A safety-oriented framework for perception uncertainty organizes its sources as conceptual uncertainty, development situation and scenario coverage, situation or scenario uncertainty, sensor properties, labeling uncertainty, model uncertainty, and operational domain uncertainty. This taxonomy places uncertainty management at the center of safety requirements: perception performance is specified as scenario-dependent uncertainty and timeliness bounds under the ODD, rather than only as aggregate detection metrics (Czarnecki et al., 2019).

Annotation quality is one of the strongest practical determinants of these bounds. A multi-organisation case study across six companies and four research institutes identified 18 recurring annotation error types across completeness, accuracy, and consistency. The study was based on 19 semi-structured interviews with 20 experts, more than 50 hours of transcripts, and dual coding with Cohen’s $$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$3. Reported failure modes included attribute omission, edge-case omission, selection bias, wrong class label, bounding-box errors, inter-annotator disagreement, ambiguous instructions, misaligned hand-offs, and cross-modality misalignment; practitioners described the taxonomy as a failure-mode catalogue similar to FMEA (Saeeda et al., 20 Nov 2025). These issues directly induce label noise, calibration drift, and domain-shift fragility.

Prototype deployment work shows that uncertainty-aware perception can be integrated into real-time systems. In transformer-based 3D detection, Platt Scaling reduced D-ECE from $$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$4 to $$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$5, and regression temperature scaling reduced MCA$$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$6 from $$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$7 to $$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$8 and MCA$$L(\theta, X, Y) = KL[q(w \mid \theta)\|p(w)] - E_{q(w\mid\theta)}[\log p(Y \mid X, w)].$$9 from $$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$0 to $$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$1. Masked-modal training improved mRRA from $$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$2 to $$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$3, and a prototype vehicle implementation achieved average inference latency of $$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$4 with a $$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$5th-percentile tail latency of $$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$6 on an RTX 4090 while visualizing attention saliency, model uncertainty state, and documentation artifacts in real time (Beemelmanns et al., 15 May 2026). A different acceleration route is provided by a photonic Bayesian machine, which offers a $$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$7 digital interface, executes probabilistic convolutions within $$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$8, and achieved AUROC $$H[p(y \mid x, D)] = - \sum_c p(y=c \mid x,D)\log p(y=c \mid x,D)$$9 for OOD blood-cell detection while improving MNIST accuracy from $$MI(y,w \mid x,D)=H[p(y \mid x,D)]-E_{p(w\mid D)}[H[p(y \mid x,w)]]$$0 to $$MI(y,w \mid x,D)=H[p(y \mid x,D)]-E_{p(w\mid D)}[H[p(y \mid x,w)]]$$1 under MI-based rejection (Brückerhoff-Plückelmann et al., 1 Dec 2025).

At the governance level, uncertainty also shapes how AI itself is perceived. Across 71 AI scenarios, experts anticipated higher probabilities, perceived lower risks, reported greater utility, and expressed more favorable sentiment than the public; in regression models of valence, the public weighted risk about half as strongly as benefit, whereas experts weighted risk about one third as strongly as benefit (Brauner et al., 2024). This does not describe perception models directly, but it clarifies a practical boundary condition: uncertainty must be calibrated not only mathematically and operationally, but also institutionally and socially. The literature therefore converges on a broad view in which AI-induced perception uncertainties are simultaneously probabilistic objects, control-relevant constraints, human-interaction variables, and assurance artifacts.