Causally-Guided Gaussian Perturbations (CGP)

• The paper’s main contribution is a ViT-guided Gaussian perturbation strategy that reduces spurious correlations by weakening background regions.
• It implements a two-stage training process where initial ViT-based causal guidance enables robust feature learning, followed by a CNN fine-tuning for deployment.
• Quantitative results on Camelyon17 show improved OOD accuracy (up to 81.94%) with only a modest drop in in-distribution performance.

Causally-Guided Gaussian Perturbations (CGP) is a lightweight framework for out-of-distribution (OOD) generalization in medical imaging that uses a Vision Transformer (ViT) to estimate likely causal regions and then injects spatially varying Gaussian noise so that foreground regions receive weak perturbation and background regions receive strong perturbation. In the formulation introduced for histopathology, the method is intended to discourage reliance on context, stain artifacts, and background textures, thereby shifting the predictor toward biologically meaningful features that remain stable across hospitals, scanners, and staining conditions. The published instantiation is evaluated on the WILDS Camelyon17 benchmark and is reported to deliver consistent OOD gains with only a modest trade-off in in-distribution (ID) accuracy (Pei et al., 30 Sep 2025).

1. Problem formulation and causal motivation

CGP addresses a standard failure mode of medical-image classifiers trained under domain shift: models that achieve strong source-domain performance often degrade when deployed on a new hospital or scanner. In histopathology, the stated sources of shift include staining, tissue preparation, and image acquisition, all of which can induce spurious correlations. The method is positioned against OOD approaches such as IRM, GroupDRO, V-REx, and IRMX, which pursue statistical domain invariance but, in the paper’s framing, do not explicitly model causal structure (Pei et al., 30 Sep 2025).

The causal intuition is spatial. The image is decomposed into causal/foreground regions, described as areas genuinely predictive of the label, and non-causal/background regions, described as contextual or spurious patterns that may correlate with the label in the training data but are unstable across domains. In the histopathology setting, the canonical foreground examples are cancer-related tissue morphology, whereas the non-causal regions include stain artifacts and background structure. CGP operationalizes this distinction by perturbing the complement of a learned causal mask more aggressively than the masked foreground.

This intervention-based view is central to the method’s rationale. Rather than regularizing representations only through domain-level objectives, CGP changes the input distribution so that non-causal regions become unreliable. The intended consequence is that the model is pushed toward stable foreground cues. The paper explicitly frames this as a causal intervention rather than pure invariance regularization (Pei et al., 30 Sep 2025).

2. Core architecture and perturbation mechanism

CGP is described as plug-and-play and is instantiated on top of FeAT, though the paper states that it can be used with other training frameworks. The end-to-end pipeline begins with a ViT encoder applied to an input image $x$, producing patch-token features

$T = \text{ViT}_{\text{enc}(x)}.$

These tokens preserve spatial structure at patch level and serve as the substrate for causal-region estimation (Pei et al., 30 Sep 2025).

A causal region prediction head then produces a Soft Causal Mask (SCM). The paper specifies this as a linear layer followed by a sigmoid, with bilinear upsampling to image resolution: $$M = \text{Upsample}\left(\sigma(W_{\text{mask}} T)\right),$$ where $M \in [0,1]^{H \times W}$. The mask is explicitly soft rather than binary, allowing perturbation magnitude to vary continuously over the image. High mask values indicate likely causal foreground and therefore weak noise; low mask values indicate likely background or spurious context and therefore strong noise (Pei et al., 30 Sep 2025).

The defining perturbation is a pixelwise, spatially varying Gaussian corruption: $$x_{\text{adv}} = x \cdot M + (1 - M)\cdot \mathcal{N}(0,\sigma^2).$$ In the paper’s interpretation, pixels in regions where $M$ is close to $1$ are kept mostly intact, whereas pixels where $M$ is close to $0$ are replaced or dominated by Gaussian noise. The asymmetry between foreground and background is the mechanism by which the intervention is made causal. The paper also states that this pixel-level perturbation is preferable to feature-level perturbation because it avoids assumptions of feature independence and is easier to interpret spatially (Pei et al., 30 Sep 2025).

3. Objective function and two-stage training strategy

Because the SCM may be unreliable early in training, CGP adds a confidence-based weighting factor $\lambda_{\text{adv}}(x)$ based on the ViT prediction confidence $T = \text{ViT}_{\text{enc}(x)}.$0. The weighting function uses a threshold $T = \text{ViT}_{\text{enc}(x)}.$1 and steepness $T = \text{ViT}_{\text{enc}(x)}.$2: $T = \text{ViT}_{\text{enc}(x)}.$3 The intended behavior is explicit: confident ViT predictions imply larger weight on the perturbed-sample loss, while uncertain predictions imply smaller weight to avoid noisy supervision. The experimental setting uses $T = \text{ViT}_{\text{enc}(x)}.$4 and $T = \text{ViT}_{\text{enc}(x)}.$5 (Pei et al., 30 Sep 2025).

The total loss combines three cross-entropy terms,

$T = \text{ViT}_{\text{enc}(x)}.$6

where $T = \text{ViT}_{\text{enc}(x)}.$7 is the classification loss on clean inputs, $T = \text{ViT}_{\text{enc}(x)}.$8 is the loss from the ViT-guided foreground-focused branch, and $T = \text{ViT}_{\text{enc}(x)}.$9 is the loss on the perturbed image $$M = \text{Upsample}\left(\sigma(W_{\text{mask}} T)\right),$$0. In the algorithmic description, predictions are made on clean input $$M = \text{Upsample}\left(\sigma(W_{\text{mask}} T)\right),$$1, masked input $$M = \text{Upsample}\left(\sigma(W_{\text{mask}} T)\right),$$2, and perturbed input $$M = \text{Upsample}\left(\sigma(W_{\text{mask}} T)\right),$$3 (Pei et al., 30 Sep 2025).

Training is split into two stages. In Stage 1, the model is trained with ViT guidance and perturbations so as to shape feature learning. In Stage 2, the ViT and perturbation modules are discarded, and only the CNN classifier is fine-tuned on clean data. The final deployed model is therefore ViT-free, with the ViT serving as a training-time causal guide rather than an inference-time dependency. A plausible implication is that the method seeks OOD robustness without increasing deployment-time complexity (Pei et al., 30 Sep 2025).

4. Experimental setting and quantitative performance

The reported evaluation is on Camelyon17, a WILDS benchmark for OOD generalization in histopathology. The task is binary classification of lymph node tissue patches for metastatic cancer detection. The dataset is derived from whole-slide images collected from five hospitals, each treated as a separate domain, and contains over 450,000 patches of size $$M = \text{Upsample}\left(\sigma(W_{\text{mask}} T)\right),$$4. Training uses domains 0–2 with 302,436 training patches, validation uses domain 3 with 34,038 ID validation patches, and testing uses domain 4 with 119,480 OOD test patches (Pei et al., 30 Sep 2025).

The experimental backbone is FeAT. Compared methods include ERM, IRM, IRMX, GroupDRO, and V-REx. Images are resized to $$M = \text{Upsample}\left(\sigma(W_{\text{mask}} T)\right),$$5, ImageNet normalization is applied, random 90-degree rotations and horizontal flips are used during training, batch size is 32, and results are averaged over 10 random seeds. For Camelyon17, the ViT is initialized from Hugging Face Transformers and trained from scratch to match the from-scratch CNN setup (Pei et al., 30 Sep 2025).

The main empirical claim is that CGP achieves the best OOD performance across all initialization strategies, with OOD accuracy between 80.68\% and 81.94\%, exceeding the best baseline OOD result of 80.41\%, while incurring only a small ID reduction.

Training objective	FeAT baseline ID / OOD	CGP ID / OOD
IRM	80.81 / 77.97	81.70 / 81.22
IRMX	82.77 / 76.91	81.68 / 80.68
GroupDRO	82.34 / 80.41	82.06 / 81.94
V-REx	82.73 / 75.12	81.70 / 81.40

These results are presented as stable across optimization choices, which the paper interprets as evidence that CGP is not overly sensitive to initialization. The contrast most emphasized in the text is GroupDRO on FeAT: the best baseline ID score is 82.77\%, whereas CGP reaches 82.06\% ID and 81.94\% OOD, improving over the baseline OOD score of 80.41\% (Pei et al., 30 Sep 2025).

5. Interpretability, localization behavior, and causal reading of the masks

The paper associates CGP’s performance gains with improved localization of medically meaningful tissue regions. Visual comparisons are reported against standard CAM, Grad-CAM, CNN-based saliency maps, and ViT-generated foreground masks. The stated qualitative finding is that the ViT masks tend to highlight more semantically meaningful cancer-related regions than standard CAM, which can be broader and less focused (Pei et al., 30 Sep 2025).

The foreground emphasis described in the paper includes dense cellular regions, irregular nuclear morphology, and strong staining patterns associated with malignancy. Regions the method tends to suppress include cytoplasm, extracellular matrix, tissue gaps, and staining artifacts. In the paper’s interpretation, this shift supports the claim that perturbing non-causal regions reduces shortcut learning and forces the classifier to depend on stable signals that transfer across domains (Pei et al., 30 Sep 2025).

This localization behavior is also the basis for the method’s interpretability claims. Because the perturbation is spatial and mask-guided, it is more directly inspectable than feature-space regularization. At the same time, the paper notes an important limitation: the masks are low-resolution and must be upsampled, which smooths fine details and affects the precision of both interpretation and perturbation localization. A plausible implication is that CGP’s causal reading is strongest at coarse regional scale rather than at fine-grained cellular boundaries (Pei et al., 30 Sep 2025).

6. Scope, limitations, and relation to adjacent methods

The limitations explicitly identified for CGP are methodological and epistemic. First, ViT-generated mask quality can vary across datasets. Second, more medical benchmarks should be tested. Third, interpretability remains constrained by low-resolution mask upsampling. Fourth, domain-specific knowledge is still needed to fully validate causal explanations. These caveats delimit the strength of the causal claims: the masks are soft causal proxies, not verified causal segmentations (Pei et al., 30 Sep 2025).

The term Causally-Guided Gaussian Perturbations should also be distinguished from other causality-and-perturbation formulations in adjacent literature. The counterfactual-explanation framework CECAS uses a causally guided adversarial steering procedure with a spurious-factor preservation loss and a causal post-refinement stage, but it is not called CGP and is directed at counterfactual visual explanations rather than OOD generalization in medical imaging (Qiao et al., 14 Jul 2025). Likewise, the Causal Gaussian Process Convolution Model (CGPCM) denotes a probabilistic model for stationary time series generated by filtering white noise through a causal moving-average filter; this is a different use of causal Gaussian structure from CGP’s spatial image perturbation mechanism (Bruinsma et al., 2018). Later work on the CGPCM and RGPCM further studied how causal convolution affects spectral decay and roughness in non-smooth signals, again in time-series modeling rather than image robustness (Bruinsma et al., 2022).

A broader neighboring line of work models perturbations through explicit structural interventions rather than Gaussian image corruption. Generative Intervention Models (GIMs) learn a mapping from perturbation features to distributions over atomic interventions in a structural causal model, with applications including scRNA-seq drug perturbation data (Schneider et al., 2024). This suggests that recent causal perturbation research spans at least three distinct regimes: spatially varying Gaussian image noise for OOD robustness, adversarially steered counterfactual editing, and perturbation-conditioned intervention modeling in structural causal systems. Within that landscape, CGP is specifically the ViT-guided, pixel-level Gaussian perturbation method introduced for medical-image OOD generalization (Pei et al., 30 Sep 2025).