Deep Spurious Regression (DSR)

• Deep Spurious Regression (DSR) is a framework that addresses continuous prediction challenges by mitigating shortcut learning through attribute–label confounding.
• It employs two-dimensional distribution smoothing along both target and attribute axes to correct for sparse or missing combinations in training data.
• The methodology combines attribute-conditional Label Distribution Smoothing with affinity-based Multi-Dimensional Scaling to improve robustness across diverse datasets.

Deep Spurious Regression (DSR) denotes the problem of learning continuous targets from data with attribute–label confounding, under conditions in which some attribute–target combinations are sparse or absent in training and the deployment distribution is balanced over the full range of targets and spurious attributes. In this setting, a regressor can exploit an attribute $a$ as a shortcut for a real-valued target $y$, because $p_{\text{train}}(y \mid a)$ is highly predictive during training, yet fail sharply when $p_{\text{test}}(y \mid a)$ changes at test time. The formulation was introduced to extend the spurious-correlation literature beyond classification, where labels are discrete and group–label pairs are natural, into continuous prediction problems such as age estimation, temperature regression, poverty prediction, and code runtime estimation (Xu et al., 1 Jun 2026).

1. Formal problem formulation

DSR is defined on training data

$\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$

where $\mathbf{x}_i \in \mathcal{X}$ denotes the input, $y_i \in \mathcal{Y}\subset \mathbb{R}$ is a continuous target, and $a_i \in \mathcal{A}$ is a spurious attribute. The predictive model is a regressor

$f_\theta:\mathcal{X}\to\mathbb{R},$

typically trained by ERM with an $\ell_1$ loss,

$y$0

The defining shift is an attribute–label shift: $y$1 By construction in the DSR setup, the attribute $y$2 is strongly predictive of $y$3 in training, while the test distribution is roughly uniform over $y$4. The objective is therefore not merely to minimize average regression error, but to generalize uniformly over all combinations $y$5, including combinations that are rare, sparse, or entirely unobserved during training.

For analysis, the continuous target space is partitioned into $y$6 non-overlapping bins $y$7. If $y$8, the sample is assigned bin index $y$9, and a group is defined as a bin–attribute pair

$p_{\text{train}}(y \mid a)$0

with group count

$p_{\text{train}}(y \mid a)$1

These groups are auxiliary partitions rather than semantic labels. Their purpose is diagnostic and operational: they expose many-shot, few-shot, and zero-shot regions of the joint target–attribute space.

The central failure mode is a fragmented, shortcut-driven mapping from $p_{\text{train}}(y \mid a)$2 to $p_{\text{train}}(y \mid a)$3. The regressor learns to partially encode $p_{\text{train}}(y \mid a)$4 as a proxy for $p_{\text{train}}(y \mid a)$5, because doing so is low-cost under the skewed training distribution. When that shortcut breaks at test time, errors can become catastrophically large in regions that were weakly represented or absent in training.

2. Why regression shortcuts are not the same as classification shortcuts

In the classical spurious-correlation setting for classification, labels are discrete, labels and attributes form a finite collection of meaningful group–label pairs, and robustness is usually formulated as improving worst-group performance. DSR departs from that template in several ways (Xu et al., 1 Jun 2026).

No natural discrete groups: with continuous targets, there are infinitely many possible values. Any discretization into bins is imposed for analysis and training, not inherent to the task.

Target continuity: nearby target values are semantically related. Unlike classification labels, regression targets possess ordering and distance structure, so information from nearby values should be shared rather than treated as categorically unrelated.

Continuous error behavior: in classification, off-diagonal group–label failure is often almost binary. In DSR, error varies smoothly with distance in target space from the attribute’s dominant training region. The paper’s comparison between classification and regression under the same spurious structure highlights that regression failure curves are graded rather than binary.

Sparsity and missing combinations: some $p_{\text{train}}(y \mid a)$6 combinations may have zero training samples. Group-based DRO or JTT do not directly optimize over groups with no observed data, whereas DSR inherently requires interpolation and extrapolation across both target and attribute dimensions.

Similarity among attributes: attributes may have similar target distributions or similar learned feature embeddings. Treating all groups as unrelated discards exploitable geometry across attributes.

A common misconception is that classification-oriented group-robust methods can be transferred to regression by simple binning. The DSR formulation rejects that reduction: binning is useful operationally, but it does not remove the need to model continuity in $p_{\text{train}}(y \mid a)$7 and similarity across $p_{\text{train}}(y \mid a)$8.

3. Methodological framework: two-dimensional distribution smoothing

The proposed strategies are organized around two-dimensional distribution smoothing. Along the target axis, the method exploits continuity of $p_{\text{train}}(y \mid a)$9 within each attribute. Along the attribute axis, it exploits similarity among attributes, measured either through label distributions or through learned representations.

The target-axis component is attribute-conditional Label Distribution Smoothing (LDS). For each attribute $p_{\text{test}}(y \mid a)$0, the empirical distribution of labels is smoothed within that attribute to obtain $p_{\text{test}}(y \mid a)$1. Each sample then receives the weight

$p_{\text{test}}(y \mid a)$2

where $p_{\text{test}}(y \mid a)$3 controls reweighting strength. This differs from the original LDS setting, which uses a global label density rather than an attribute-conditional one.

The attribute-axis component introduces an affinity matrix $p_{\text{test}}(y \mid a)$4, where $p_{\text{test}}(y \mid a)$5 specifies how much the counts of attribute $p_{\text{test}}(y \mid a)$6 should contribute to attribute $p_{\text{test}}(y \mid a)$7. Using group counts $p_{\text{test}}(y \mid a)$8, the smoothed group count is

$p_{\text{test}}(y \mid a)$9

and the corresponding attribute-based weight is

$\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$0

The combined weight is

$\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$1

which yields the weighted regression objective

$\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$2

The key technical question is how to construct $\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$3. The paper uses Multi-Dimensional Scaling (MDS) followed by an RBF kernel. A pairwise distance matrix $\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$4 over attributes is first computed, classical MDS embeds attributes into a 2D Euclidean space $\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$5, and the affinity is defined by

$\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$6

with $\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$7 set to the median pairwise distance in $\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$8.

Two variants are proposed. L-MDS constructs $\mathcal{D}=\{(\mathbf{x}_i,y_i,a_i)\}_{i=1}^{N},$9 from Wasserstein-1 distances between per-attribute label distributions,

$\mathbf{x}_i \in \mathcal{X}$0

Because Wasserstein-1 respects label geometry, nearby supports in target space remain close even without exact overlap. L-MDS is precomputed before training and incurs no additional runtime overhead. F-MDS instead derives $\mathbf{x}_i \in \mathcal{X}$1 from learned feature centroids. If $\mathbf{x}_i \in \mathcal{X}$2 is the normalized encoder feature, then each attribute centroid is

$\mathbf{x}_i \in \mathcal{X}$3

and distances are

$\mathbf{x}_i \in \mathcal{X}$4

F-MDS is updated every $\mathbf{x}_i \in \mathcal{X}$5 epochs and is therefore adaptive to the evolving representation geometry.

The training procedure bins the target space, computes attribute-conditional LDS, constructs a static kernel for L-MDS or periodically updates a kernel for F-MDS, computes weights $\mathbf{x}_i \in \mathcal{X}$6 and $\mathbf{x}_i \in \mathcal{X}$7, and optimizes the weighted $\mathbf{x}_i \in \mathcal{X}$8 loss. The paper also notes that L-MDS and F-MDS can be combined by averaging or composing their kernels or weights.

4. Geometric interpretation and theoretical status

The paper’s theoretical contribution is deliberately limited. It does not provide formal theorems on convergence or robustness bounds; instead, it offers a geometric and empirical account of why DSR requires different machinery from classification (Xu et al., 1 Jun 2026).

Two conceptual observations structure that account. The first is target continuity within attribute, which motivates within-attribute smoothing along the target axis. The second is similarity among attributes, which motivates smoothing across attributes rather than treating them as disjoint domains.

The paper’s toy example, ColoredRotatedMNIST, illustrates this geometry. Each background color acts as a spurious attribute and is associated in training with a dominant rotation-angle range. Under ERM, test error across angle forms a curve rather than a set of discrete failures. Some sparse regions have low error, while others fail sharply when the color–angle shortcut breaks. This directly supports the claim that low data density does not always imply high error; what matters is whether the shortcut remains locally predictive.

A further geometric interpretation appears in representation space. When attributes have similar label distributions, ERM feature embeddings of groups overlap more; when those distributions diverge, embeddings separate. This suggests that feature-space structure can be used to infer attribute affinities, which is precisely the role assigned to F-MDS.

The absence of formal guarantees is itself part of the current state of the field. DSR, as defined here, is presented primarily as a benchmark and methodological framework: a way to expose the inadequacy of purely discrete group-based robustness notions for continuous prediction, and to replace them with smoothing mechanisms that respect the geometry of both $\mathbf{x}_i \in \mathcal{X}$9 and $y_i \in \mathcal{Y}\subset \mathbb{R}$0.

5. Benchmarks, datasets, and empirical behavior

The study instantiates DSR on four datasets spanning computer vision, environmental sensing, and LLM-based regression (Xu et al., 1 Jun 2026).

Dataset	Continuous target	Spurious attribute
UTKFace	Age (continuous, 1–116)	Race/ethnicity: White, Black, Asian, Indian, Others
SkyFinder	Temperature (continuous, $y_i \in \mathcal{Y}\subset \mathbb{R}$1 to $y_i \in \mathcal{Y}\subset \mathbb{R}$2C)	Camera ID
PovertyMap	Poverty index, continuous in roughly $y_i \in \mathcal{Y}\subset \mathbb{R}$3	Country
CodeNet	CPU runtime (ms), clamped to $y_i \in \mathcal{Y}\subset \mathbb{R}$4	Programming language

The data splits are 17 620 train, 2 753 val, 3 730 test for UTKFace; 64 945 train, 9 335 val, 6 766 test for SkyFinder; 6 034 train, 475 val, 545 test for PovertyMap; and 19 500 train, 6 374 val, 6 374 test for CodeNet. UTKFace, SkyFinder, and PovertyMap use a ResNet-18 backbone. CodeNet uses RLM-GemmaS-Code-V0, with the encoder frozen and the decoder trained. Optimization is standard, with SGD or AdamW, a cosine learning-rate schedule, 400 epochs for vision, and 20 epochs for the LLM model.

Baselines include ERM, inverse-frequency reweighting, square-root reweighting, CBLoss, global LDS, RnC, DANN, and GroupDRO. Evaluation uses Mean Absolute Error (MAE), the Geometric Mean of error (GM), attribute-wise average and worst MAE, and shot-wise average and worst MAE. Shot partitions are defined over $y_i \in \mathcal{Y}\subset \mathbb{R}$5 groups as many-shot $y_i \in \mathcal{Y}\subset \mathbb{R}$6, medium-shot $y_i \in \mathcal{Y}\subset \mathbb{R}$7–$y_i \in \mathcal{Y}\subset \mathbb{R}$8, few-shot $y_i \in \mathcal{Y}\subset \mathbb{R}$9, and zero-shot $a_i \in \mathcal{A}$0 for UTKFace, SkyFinder, and PovertyMap.

The ERM results establish the severity of the problem. On UTKFace, overall MAE is 7.39 while zero-shot worst MAE is 73.56. On SkyFinder, overall MAE is 3.68 and zero-shot worst MAE is 29.78. On PovertyMap, overall MAE is 0.504 and zero-shot worst MAE is 1.996. On CodeNet, overall MAE is 268.7 and few-shot worst MAE is 711.3. These gaps quantify the paper’s notion of catastrophic DSR failure: average error can appear moderate while sparse or unseen target–attribute regions collapse.

Across the benchmark suite, L-MDS and F-MDS consistently improve overall MAE over ERM and other baselines, with the largest gains in few-shot and zero-shot regions. On UTKFace, the best model is F-MDS, reducing overall MAE from 7.39 to 7.22, zero-shot worst MAE from 73.56 to 68.81, and few-shot average MAE from 7.19 to 6.71. On PovertyMap, L-MDS is best, improving overall MAE from 0.504 to 0.486 and zero-shot average MAE from 0.744 to 0.720. On CodeNet, L-MDS reduces overall MAE from 268.7 to 243.4, while few-shot average MAE improves from 529.8 to 440.2 with L-MDS or 429.0 with F-MDS. On SkyFinder, SqrtReWeight or RnC performs best on overall MAE, but L-MDS and F-MDS remain strong in sparse regions; few-shot average MAE improves from 4.49 to 4.17, and zero-shot average MAE from 5.22 to 4.74 with F-MDS.

The aggregate ranking reinforces the dataset-level picture. Average rank over all datasets and metrics is 3.55 for L-MDS, 3.68 for F-MDS, and 6.41 for ERM, while DRO and DANN rank worse. Additional analyses show that on a curated UTKFace subset with missing age intervals and truncated extremes, L-MDS improves MAE in zero-shot target regions by up to 4.82 absolute points and GM by 3.84, and that the gains of L-MDS and F-MDS over ERM increase as the amount of UTKFace training data is reduced down to 10%.

6. Practical implications, limitations, and position in the literature

Practically, DSR reframes robust regression under spurious correlations as a problem of modeling joint imbalance in $a_i \in \mathcal{A}$1 rather than marginal imbalance in $a_i \in \mathcal{A}$2 alone. The paper’s recommendations follow directly from this reframing: make spurious attributes available as metadata during training, exploit label continuity with attribute-conditional LDS, exploit attribute similarity with an affinity matrix built from label distributions or feature embeddings, train with weighted regression losses, and evaluate worst-case and shot-wise performance rather than average MAE alone (Xu et al., 1 Jun 2026).

The framework also clarifies several boundaries of current methods. It assumes attribute labels are available at training time and does not address latent attribute discovery. It does not formally control the trade-off between many-shot and few/zero-shot performance. Its MDS-based kernels are intentionally simple. It offers no formal robustness guarantees. Extending DSR to partially observed or unknown attributes, and to other model types such as time series or structured outputs, remains open.

Within the broader literature, DSR generalizes the spurious-correlation perspective from discrete prediction to continuous prediction. Methods such as GroupDRO, JTT, LfF, IRM, DFR, and CNC typically assume discrete labels and finite group–label pairs. DANN and related domain-invariant methods aim to remove domain information, whereas the DSR view treats the attribute as not purely nuisance: its geometry in label and feature space can be informative for smoothing and calibration. Relative to imbalanced-regression methods such as LDS, BalancedMSE, MDLT, and RnC, DSR emphasizes that marginal label imbalance is not the same as attribute–label confounding. Relative to causal inference, the framework remains pragmatic rather than fully causal: it adopts a shortcut-learning perspective and does not perform explicit causal adjustment.

A plausible implication is that DSR marks a shift in how robustness is defined for regression. Instead of worst-group accuracy over a finite taxonomy, the target becomes stable performance over a continuous target manifold crossed with a spurious-attribute space. In that sense, DSR is less a minor extension of classification robustness than a distinct problem class with its own benchmarks, failure modes, and algorithmic design principles.