D-GAP: Dataset-Agnostic Gradient Augmentation

• The paper demonstrates that D-GAP achieves state-of-the-art OOD performance by combining gradient-driven amplitude perturbation with targeted pixel blending.
• D-GAP integrates Fourier space mixing guided by task gradients to mitigate frequency shortcut learning and promote robust spectral representations.
• The dual-space fusion approach preserves detailed spatial information while delivering significant accuracy and macro-F1 gains across diverse benchmarks.

D-GAP (Dataset-agnostic and Gradient-guided Augmentation in Amplitude and Pixel spaces) is an augmentation framework for out-of-domain (OOD) robustness in computer vision, which integrates targeted augmentation in both frequency and pixel spaces. D-GAP uniquely computes sensitivity maps in the amplitude domain via task gradients and fuses these augmented images with pixel-level blends, thereby reducing frequency-based shortcut learning and preserving spatial detail. This approach is designed to be fully dataset-agnostic and achieves state-of-the-art OOD performance across a range of real-world and benchmark datasets (Wang et al., 14 Nov 2025).

1. Motivation and Background

The challenge of OOD robustness in vision emerges from real-world distribution shifts, such as varied backgrounds (camera trap imagery), differing acquisition instruments (microscopy, telescopes), or protocol changes (histopathology stain variations). Empirical Risk Minimization (ERM)-trained networks exhibit marked drops in accuracy and macro-F1 when moved across such domains. Recent literature demonstrates that convolutional networks often exhibit frequency bias, relying disproportionately on a small set of dataset-specific frequencies termed "spectral shortcuts" (Pinson et al. 2023; He et al. 2024). When spectral statistics differ (e.g., due to new backgrounds or sensors), this bias leads to poor generalization.

Generic augmentations—RandAugment, CutMix, FACT, SAM—offer only modest and inconsistent OOD gains. Conversely, dataset-specific augmentations demand manual, task-dependent analysis and do not generalize. A common alternative, amplitude spectrum perturbation, randomizes style and global texture but can introduce blurring and ignore spatial localization. D-GAP addresses both issues via principled, gradient-driven mixing in Fourier space complemented by pixel-wise detail restoration.

2. D-GAP Pipeline

The D-GAP procedure operates on each training batch, and for every source image $X_s$ it samples a random "target-domain" image $X_t$ from a held-out pool. D-GAP then:

1. Computes a gradient-guided mix in the Fourier amplitude space to create a frequency-augmented view $\hat X_f$.
2. Synthesizes a complementary pixel-space blend $\hat X_p$.
3. Linearly fuses these ($\hat X_f$, $\hat X_p$) into the final augmentation $\tilde X$ using a dual-space fusion coefficient.
4. Augments training by feeding $\tilde X$ through the network and backpropagating on the task loss $\mathcal{L}$.

The augmentation is dynamically integrated after batch formation, immediately prior to the forward pass. On real-world tasks, D-GAP is used in a two-stage "linear-probe then fine-tune" (LP-FT) schedule, while domain generalization benchmarks employ end-to-end fine-tuning.

3. Gradient-Guided Amplitude-Space Augmentation

Let $X_s$ and $X_t$ denote source and target images, respectively, and $\mathcal{F}(\cdot)$ the 2D discrete Fourier transform. The amplitude spectra $A_s(f)=|\mathcal{F}(X_s)(f)|$ and $A_t(f)=|\mathcal{F}(X_t)(f)|$ are defined for frequency bins $f$. For model parameters $\theta$ and labels $y$, D-GAP computes:

• The sensitivity map in frequency space as the absolute gradient of the loss w.r.t. the source amplitude:

$$S(f) = \left|\frac{\partial \mathcal{L}(\theta; X_s, y)}{\partial A_s(f)}\right|$$

• Sensitivity normalization to $[0,1]$:

$\alpha(f) = \frac{S(f)}{\max_{f'} S(f')}$

• Amplitude interpolation:

$$\widetilde{A}(f) = \alpha(f)A_s(f) + (1-\alpha(f))A_t(f)$$

Frequencies with highest sensitivity ($\alpha(f) \approx 1$) are sourced from $X_s$; those less sensitive are injected from $X_t$.

• Inverse Fourier reconstruction using the original source phase $\Phi_s(f)$:

$$\mathcal{F}_\text{mix}(f) = \widetilde{A}(f) e^{j\Phi_s(f)}, \quad \hat X_f = \mathcal{F}^{-1}(\mathcal{F}_\text{mix})$$

This targeted frequency-space blending reduces spectral shortcut learning and forces the network to utilize more robust spectral patterns.

4. Pixel-Space Augmentation and Dual-Space Fusion

To counteract the loss of spatial detail from amplitude mixing, D-GAP introduces a pixel-space blend,

$\hat X_p = \beta \odot X_s + (1-\beta) \odot X_t$

where $\beta$ is either a scalar mixing ratio $\lambda_1$ (MixUp-style) or a spatial mask (optionally derived from per-pixel sensitivity, such as $|\partial\mathcal{L}/\partial X_s|$).

The final augmentation fuses both views:

$$\tilde X = (1-\lambda_2)\hat X_f + \lambda_2\hat X_p$$

with $\lambda_2 \in [0,1]$ balancing frequency and pixel contributions. This dual-space approach ensures that frequency bias is mitigated while fine image details and edges are preserved.

5. Implementation Summary

The algorithm operates per training batch via the following steps:

Step	Operation	Output
1	Sample $X_s$, $y_s$; sample $X_t$	Inputs
2	Compute task loss $\mathcal{L}$	Scalar loss
3	FFT to obtain $A_s$, $\Phi_s$, $A_t$	Spectra, phases
4	Compute $S(f)$ for $f\in \Omega_r$	Sensitivities
5	Normalize to get $\alpha(f)$	Mixing weights
6	Construct $\widetilde{A}(f)$	Mixed amplitude
7	Inverse FFT to yield $\hat X_f$	Augmented image
8	Pixel blend for $\hat X_p$	Augmented image
9	Fuse $$\tilde X = (1-\lambda_2)\hat X_f + \lambda_2\hat X_p$$	Final image
10	Forward $\tilde X$, backpropagate	Update $\theta$

Hyperparameters $\lambda_1$ and $\lambda_2$ regulate the pixel and frequency blend ratios. The sensitivity map is computed within a selected frequency region $\Omega_r$. D-GAP incurs a training overhead of approximately 10–20% due to additional gradient computation.

6. Empirical Performance and Ablation Analysis

D-GAP was extensively evaluated on both real-world OOD datasets (iWildCam, Camelyon17, BirdCalls, Galaxy10 DECaLS) and established domain generalization benchmarks (PACS, Office-Home, Digits-DG), using ResNet-50 encoders pretrained on ImageNet. Optimization employed SGD with learning rates around $1\times10^{-3}$, weight decay near $1\times10^{-4}$, and batch size 64; metrics were macro-F1 for class imbalance and accuracy otherwise.

Key empirical results include:

Dataset	Metric	Best Baseline	D-GAP	Gain
iWildCam	F₁	34.7	36.8	+2.1
Camelyon17	Acc	92.2	96.4	+4.2
BirdCalls	F₁	35.1	40.7	+5.6
Galaxy10	Acc	74.1	83.4	+9.3
PACS	Acc	87.88 (FACT)	88.47	+0.59
Office-Home	Acc	66.75 (SAM)	70.03	+3.28
Digits-DG	Acc	82.1 (SAM)	83.6	+1.5

Ablation studies show:

• Pixel-only augmentation degrades OOD performance (–6 to –20 percent).
• Frequency-only mixing offers strong gains (+2 to +4 percent), but is inferior to the full D-GAP pipeline.
• Unguided frequency mix (fixed $\alpha$) produces smaller improvement (+1 to +3 percent).
• Full D-GAP (gradient-guided $\alpha$ and pixel fusion) achieves highest OOD gains across all tasks.

7. Analytical Insights and Future Directions

D-GAP achieves a reduction in spectral shortcut bias by identifying and perturbing frequency components with high task gradient sensitivity ($|\partial\mathcal{L}/\partial A_s(f)|$), compelling networks to learn more robust and transferable spectral representations. The pixel-space blending compensates for spatial blurring and restores high-frequency details, critical for maintaining edge and textural fidelity.

Connectivity analysis using the framework of Shen et al. (2022) reveals that D-GAP substantially increases cross-domain, same-class connectivity ($\alpha/\gamma$), while maintaining moderate between-class connectivity—these effects are positively correlated with improved OOD accuracy.

Known limitations include the computational overhead for gradient-based sensitivity estimation and the necessity to tune two mixing hyperparameters ($\lambda_1$, $\lambda_2$), both of which exhibit robust ranges. Plausible future directions involve lightweight sensitivity estimation (e.g., historical gradients), integration with self-supervised and transformer-based architectures, and extension to zero-shot/few-shot cross-modal adaptation.

In summary, D-GAP delivers an automated, dataset-agnostic augmentation strategy that exploits model-informed Fourier perturbation and pixel-wise detail restoration, consistently surpassing generic and handcrafted augmentations for OOD robustness (Wang et al., 14 Nov 2025).