DSU: Domain Shifts with Uncertainty

• Domain Shifts with Uncertainty (DSU) is a method that models feature statistics as uncertain random variables, enabling exposure to potential unseen domains.
• It employs stochastic feature-space augmentation by perturbing channel-wise means and standard deviations based on intra-batch variability.
• Extensions like DSU++ and graph-based formulations demonstrate significant performance gains in diverse tasks such as classification, segmentation, and open-set fault diagnosis.

Searching arXiv for recent and foundational papers on DSU and uncertainty-aware domain shift methods. Domain Shifts with Uncertainty (DSU) denotes a class of methods that model distribution shift and uncertainty jointly rather than treating uncertainty as a purely downstream confidence estimate. In the canonical formulation for domain generalization, intermediate feature statistics—especially channel-wise means and standard deviations—are treated as uncertain random variables, and training is performed under sampled statistic perturbations that approximate unseen domains (Li et al., 2023, Li et al., 2022). In a broader usage, the same design principle appears in uncertainty-aware unsupervised domain adaptation, test-time training, federated active learning, and open-set fault diagnosis, where uncertainty is used to regulate feature learning, structural aggregation, calibration, or adaptation under shift (Lee et al., 2022, Zhu et al., 5 Jul 2026).

1. Canonical definition and mathematical object of DSU

The most explicit definition of DSU appears in the literature on domain generalization and out-of-distribution generalization through uncertain feature statistics. The central observation is that feature statistics such as channel-wise mean and standard deviation carry domain characteristics—style, texture, contrast, illumination, and related factors—and that conventional methods usually treat these statistics as deterministic values measured from current features. DSU instead assumes that, under potential domain shifts, these statistics should be modeled as uncertain distributions (Li et al., 2023).

For an intermediate feature tensor $x \in \mathbb{R}^{B \times C \times H \times W}$, DSU computes instance-wise channel statistics

$\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$

$$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$

The uncertainty associated with these statistics is estimated non-parametrically from within-mini-batch variation: $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$

$$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$

This construction interprets large cross-instance variability in a channel’s statistics as a proxy for greater potential movement of that statistic under unseen domain conditions (Li et al., 2023, Li et al., 2022).

DSU then models the mean and standard deviation as Gaussian random variables. Using reparameterization,

$$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$

$$\gamma(x) = \sigma(x) + \epsilon_{\sigma}\,\Sigma_{\sigma}(x),\quad \epsilon_{\sigma}\sim\mathcal N(\mathbf 0,\mathbf I).$$

The sampled statistics define an AdaIN-style feature transform

$$\hat{x} = \gamma(x)\,\frac{x - \mu(x)}{\sigma(x)} + \beta(x).$$

The practical interpretation is direct: DSU synthesizes statistic perturbations around the current feature state, thereby exposing the model to a neighborhood of plausible unseen domains instead of only the observed source statistics (Li et al., 2023).

2. Training-time stochasticity and inference-time calibration

In its standard form, DSU is a plug-in feature-space augmentation module. It introduces no new loss term and no extra trainable parameters; training proceeds with the usual task objective, while the stochastic statistic transformation is inserted after selected layers (Li et al., 2022). Reported insertion positions in ResNet-style backbones include the first convolution, max pooling, and successive residual stages, and applying DSU at multiple depths yields the strongest performance because different layers encode different levels of style and content (Li et al., 2023).

The training procedure is probabilistic but simple. With probability $p$, the module computes $\mu(x)$, $\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$0, estimates $\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$1, $\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$2, samples $\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$3 and $\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$4, and replaces $\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$5 by $\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$6; otherwise it acts as the identity. Reported operating points include $\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$7 for most tasks and $\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$8 for ImageNet training (Li et al., 2022). This stochasticity acts as feature-space augmentation centered on the sample’s own statistics rather than on statistics borrowed from another instance.

A later extension, often described as DSU++, adds instance-wise test-time calibration without updating network weights. For each DSU layer, the training distribution is summarized by channel-wise centers $\mu(x)_{b,c}=\frac{1}{HW} \sum_{h,w} x_{b,c,h,w},$9 and uncertainty scales $$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$0, which define “shift regions”

$$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$1

$$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$2

At inference, if a test instance lies outside these regions, its statistics are pulled back toward the training region using

$$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$3

$$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$4

with $$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$5. Reported effective settings are $$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$6 and $$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$7 (Li et al., 2023). This suggests a two-stage DSU interpretation: stochastic exposure to uncertain domains during training, followed by deterministic correction when a test instance falls beyond the learned uncertainty envelope.

3. Theoretical interpretation and empirical behavior

The DSU literature provides two main theoretical interpretations. First, the 2023 formulation gives a feature-space generalization bound in which target error depends on a Wasserstein term between source and target feature distributions. Under Gaussian feature distributions, this Wasserstein distance is explicitly tied to differences in feature means and covariances, which makes the manipulation of feature statistics central rather than incidental (Li et al., 2023). The intended implication is that broadening training-time exposure to statistic perturbations can reduce feature-level domain discrepancy to future targets.

Second, DSU admits an implicit-regularization view. In a simplified linear analysis, the expected loss under DSU perturbations equals the base loss plus regularization terms weighted by $$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$8 and $$\sigma^2(x)_{b,c}=\frac{1}{HW} \sum_{h,w} (x_{b,c,h,w} - \mu(x)_{b,c})^2.$$9. Channels with large uncertainty and large importance are penalized more heavily, pushing the model toward feature representations with more stable statistics across samples and domains (Li et al., 2023). This suggests that DSU is not merely a data augmentation heuristic; it is also a variance-sensitive regularizer on domain-dependent channels.

Empirically, the original DSU reports broad gains across vision tasks. On PACS classification with ResNet-18, the baseline average accuracy is reported as $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$0, DSU reaches $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$1, and DSU++ reaches $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$2. On GTA5 $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$3 Cityscapes semantic segmentation, the baseline mIoU is $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$4, DSU reaches $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$5, and DSU++ reaches $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$6. On cross-dataset person ReID, Market $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$7 Duke improves from $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$8 mAP/Rank-1 to $$\Sigma^2_{\mu}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\mu(x_i)_c - \mathbb{E}_b[\mu(x)_c]\big)^2,$$9 with DSU and $$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$0 with DSU++; Duke $$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$1 Market improves from $$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$2 to $$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$3 and $$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$4 (Li et al., 2023). These numbers support the claim that DSU is particularly effective when the dominant shift manifests through appearance or feature-statistic variation.

4. Broader uncertainty-aware formulations under the DSU view

Beyond the original feature-statistic formulation, later work uses uncertainty as the main signal for measuring, correcting, or exploiting domain shift. In this broader interpretation, DSU becomes a design pattern rather than a single module: uncertainty is estimated explicitly, and then fed back into alignment, calibration, or adaptation.

Paper	Shift setting	Uncertainty mechanism
"Unsupervised Domain Adaptation Based on the Predictive Uncertainty of Models" (Lee et al., 2022)	UDA	MC-dropout predictive variance
"Learning Calibrated Uncertainties for Domain Shift" (Wang et al., 2020)	Covariate shift, UDA, SSL	Density-ratio-adjusted softmax
"Uncertainty-aware Test-Time Training (UT$$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$5)" (Upadhyay, 3 Sep 2025)	Continuous test-time shift	Heteroscedastic Gaussian entropy

In MUDA, predictive variance estimated by Monte Carlo dropout is used as a surrogate for domain divergence. The method minimizes model uncertainty on target samples while preserving low source error, thereby encouraging domain-invariant and class-discriminative representations (Lee et al., 2022). A notable theoretical caveat is explicit in the paper: replacing the supremum disagreement in Ben-David-style analysis by an expectation-based quantity is empirically effective but no longer yields the original formal upper bound.

In the distributionally robust calibration line, the density ratio $$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$6 is estimated by a domain classifier and used as a sample-wise temperature on logits,

$$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$7

or in the class-regularized form given in the paper. Here, uncertainty is calibrated directly by how close a target sample is to the source distribution, rather than by posterior sampling or feature-statistic perturbation (Wang et al., 2020). This suggests a different DSU axis: domain shift is encoded as density-ratio-modulated confidence.

UT$$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$8 shifts the emphasis to online adaptation in dense regression. It uses a heteroscedastic Gaussian masked-autoencoding head, computes per-frame entropy from predicted variance, and triggers test-time training only when the entropy exceeds a source-domain quantile threshold (Upadhyay, 3 Sep 2025). In this case, uncertainty is not used to reshape the representation during offline training alone; it acts as a control signal that decides when adaptation should occur in a continuous domain-shift stream.

5. Open-set, graph-based, and structured instantiations

A particularly explicit structured instantiation appears in "Physics-Informed Graph Learning with Uncertainty Awareness for Open-Set Domain Generalization in Fault Diagnosis" (Zhu et al., 5 Jul 2026). The paper formulates rotating-machinery fault diagnosis as open-set domain generalization (OSDG), with labeled source-domain data

$$\Sigma^2_{\sigma}(x)_c = \frac{1}{B}\sum_{i=1}^{B}\big(\sigma(x_i)_c - \mathbb{E}_b[\sigma(x)_c]\big)^2.$$9

unlabeled target-domain data

$$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$0

and target labels drawn from

$$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$1

under the shift condition

$$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$2

The paper identifies three uncertainty stages: perceptual uncertainty at the feature level, structural uncertainty in graph construction, and decision uncertainty at the classifier or rejection stage (Zhu et al., 5 Jul 2026).

Its PGU-OD framework addresses these three stages with corresponding mechanisms. First, a physics-informed spectral attention module uses learnable Morlet-wavelet filters and spectrum-aware attention to stabilize spectral features against speed- and load-induced frequency shifts. Second, known classes are modeled as isotropic Gaussians $$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$3, and the class scales $$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$4 are used to define a heteroscedastic graph kernel

$$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$5

so that uncertain nodes are broadly connected but have diffused influence after normalization. Third, class prototypes and scales shape adaptive boundaries through Gaussian intra-class and inter-class losses, and test-time rejection uses both a relative prototype-distance criterion and an absolute $$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$6-style criterion (Zhu et al., 5 Jul 2026).

This formulation widens the meaning of DSU beyond feature-statistic perturbation. Here the uncertainty variables are not only channel statistics but also class dispersions, graph edge weights, and open-set rejection thresholds. The paper reports that on CWRU tasks P1–P8, PGU-OD often achieves H-scores above $$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$7–$$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$8, and that it improves especially on hard Paderborn tasks where domain shifts are severe (Zhu et al., 5 Jul 2026). A plausible implication is that once DSU is treated as a pipeline principle—perception, structure, and decision all uncertainty-aware—it becomes compatible with open-set and physics-informed settings that are far removed from the original image-style domain-generalization context.

6. Interpretation, misconceptions, and limitations

A common misconception is that DSU refers to a single Bayesian recipe. The literature does not support that reading. The original DSU formulation is parameter-free and uses Gaussian perturbations of feature statistics estimated from batch variability (Li et al., 2022). MUDA estimates epistemic uncertainty by MC dropout (Lee et al., 2022). UT$$\beta(x) = \mu(x) + \epsilon_{\mu}\,\Sigma_{\mu}(x),\quad \epsilon_{\mu}\sim\mathcal N(\mathbf 0,\mathbf I),$$9 uses heteroscedastic Gaussian aleatoric uncertainty for test-time scheduling (Upadhyay, 3 Sep 2025). PGU-OD is described as non-Bayesian but probabilistic, with learned Gaussian class scales that enter graph learning and open-set boundaries (Zhu et al., 5 Jul 2026). Across these works, the shared principle is not a particular posterior approximation but the explicit use of uncertainty as a structural variable under shift.

Another misconception is that DSU is equivalent to calibration. Calibration is central in some methods, especially those based on density-ratio-adjusted softmax (Wang et al., 2020), but the original DSU papers primarily target robustness by perturbing feature statistics during training (Li et al., 2023). Conversely, DSU is not restricted to covariate-style appearance shifts. The original feature-statistic model is most naturally suited to style-like or statistics-driven domain shifts and explicitly notes limitations for semantic or label-level changes (Li et al., 2023). Open-set extensions such as PGU-OD broaden the scope to unknown classes, while active and federated evidential approaches in related work use epistemic uncertainty to prioritize informative samples under multi-domain mismatch. This suggests that “DSU” now functions both as a named method and as a wider methodological viewpoint.

The main limitations stated across the literature are consistent. The feature-statistic formulation assumes Gaussianity with diagonal covariance and relies on batch-statistic estimates that may be noisy for small batch sizes (Li et al., 2023). Density-ratio approaches are grounded in covariate-shift assumptions and may weaken when label or conditional shift is strong (Wang et al., 2020). MUDA’s expectation-based disagreement surrogate relaxes formal guarantees from classical UDA theory (Lee et al., 2022). UT$$\gamma(x) = \sigma(x) + \epsilon_{\sigma}\,\Sigma_{\sigma}(x),\quad \epsilon_{\sigma}\sim\mathcal N(\mathbf 0,\mathbf I).$$0 assumes smoothly evolving domains and depends on the informativeness of the self-supervision head (Upadhyay, 3 Sep 2025). These limitations do not invalidate DSU; they delimit which uncertainty source is being modeled and which shift family is being targeted.

Taken together, the literature defines DSU as a technically precise response to a recurrent failure mode of modern learning systems: under domain shift, accuracy degrades, but so does the faithfulness of confidence. The canonical answer is to build uncertainty into the representation itself, whether by randomizing feature statistics, parameterizing class dispersion, estimating predictive variance, or modulating adaptation schedules. The unifying aim is not merely to survive domain shift, but to do so with uncertainty that remains operationally meaningful under that shift (Li et al., 2023, Zhu et al., 5 Jul 2026).