ResAD: Residual-Based Deep Learning Frameworks

• ResAD is a family of machine learning frameworks that predict deviations from normal references to enhance robustness and interpretability.
• The approach employs tailored normalization, residual diffusion decoders, and anomaly scoring to achieve transferable inference across varied domains.
• Empirical results demonstrate competitive performance in autonomous driving, cross-class anomaly detection, and fundus imaging with reduced false positives.

ResAD refers to a family of machine learning frameworks and algorithms that leverage residual modeling for robust, flexible, and generalizable inference across domains. Although the acronym appears in several contexts—including trajectory prediction for autonomous driving, class-generalizable anomaly detection in vision, and region-aware anomaly localization in medical imaging—the central unifying principle is prediction, transformation, or detection in the space of residuals with respect to suitably chosen references or normal exemplars. This entry focuses predominantly on the technical foundations, architectures, and empirical performance of leading ResAD variants as presented in recent literature.

1. Motivation and Problem Definitions

Residual-based modeling in deep learning is motivated by challenges specific to each domain:

• End-to-End Autonomous Driving: Direct regression of future trajectories from sensory input suffers from spatio-temporal target imbalance and encourages spurious, non-causal correlations. Specifically, distant future waypoints (due to their higher uncertainty and larger errors) dominate loss signals, distracting models from near-term, safety-critical planning. Naively learning the mapping from scene observations to trajectories may result in models easily overfitting nuisance correlations rather than underlying driving causality (Zheng et al., 9 Oct 2025).
• Class-Generalizable Anomaly Detection: Conventional anomaly detectors rely on the learned distribution of features tightly coupled to known classes. As a result, their application to novel classes (without retraining) leads to severe degradation. The core issue is that feature representations themselves are highly class-specific, impeding transfer (Yao et al., 2024).
• Region and Spatial Aware Anomaly Detection: In retinal fundus imaging, pixel-level anomaly detection is hampered by high anomaly scores around normal, salient structures (e.g., vessels, optic discs). This results from limited harvesting of regional context or long-range dependencies, leading to elevated false positives in normal tissue (Niu et al., 2023).

Residual modeling addresses these by reframing the target as a deviation—either from a deterministic reference (e.g., inertial physics for trajectories) or from local normality (e.g., closest patch features)—to isolate the core factors of interest: causality in driving, distributional shift in anomaly detection, or pathological context in medical images.

2. Core Methodological Frameworks

ResAD frameworks instantiate the residual paradigm through distinct but conceptually parallel architectures.

2.1. Normalized Residual Trajectory Modeling for Autonomous Driving

ResAD for trajectory prediction constructs a deterministic inertial "counterfactual"—the default constant-velocity path—at each step, then learns the (normalized) residual needed to match ground-truth waypoints: $$\mathbf{r}_t = \mathbf{p}^{gt}_t - \mathbf{p}^{0}_t$$ where $\mathbf{p}_t^{0}$ is obtained by constant-velocity extrapolation from current position $\mathbf{p}_0$ and velocity $\mathbf{v}_0$.

Residuals are component-wise normalized over the dataset to $[-\gamma, \gamma]$: $$\tilde{r}_t^d = 2\gamma\,\frac{r_t^d - r_{\min}^d}{r_{\max}^d - r_{\min}^d + \epsilon} - \gamma$$ To generate diverse, plausible futures, the reference can be perturbed in velocity space, spawning multiple hypotheses.

2.2. Residual Feature Learning for Class-Generalizable Anomaly Detection

Here, residuals are defined in feature space: for each spatial feature $x$ in an image $I$, the closest "normal" patch $x_n^*$ from a reference pool is located, and the residual $r = x - x_n^*$ is computed. This cancels class-specific information, yielding a distribution more invariant between classes: $\mathbf{p}_t^{0}$0 These residuals are mapped by a shallow constraintor $\mathbf{p}_t^{0}$1 into a hypersphere around the origin for all normal data, while anomalies remain outside, enforced by a specialized OCC loss. A flow-based density estimator then models this constrained distribution for out-of-distribution detection (Yao et al., 2024).

2.3. Region and Spatial-Aware Residuals for Fundus Anomaly Detection

In the ReSAD framework, multi-level features are composed via a region-aware (local, distance-decayed context) and a spatial-aware (self-attention, long-range correlation) operator to form pixel-level descriptors: $\mathbf{p}_t^{0}$2 where $\mathbf{p}_t^{0}$3 is a distance-filtered local aggregation, and $\mathbf{p}_t^{0}$4 reweights features by self-attention affinities. A compressed memory bank of normal pixel descriptors enables efficient nearest-neighbor distance computation for anomaly scoring (Niu et al., 2023).

3. Training Objectives, Losses, and Normalization Schemes

The residual learning paradigm is distinguished by (1) per-target or per-feature normalization and (2) tailored loss constructions to achieve robust optimization and stable transfer.

• Trajectory Domain: Point-wise residual normalization (PRNorm) ensures scale uniformity across all future waypoints:

$\mathbf{p}_t^{0}$5

Training proceeds in a conditional diffusion setting, with meansquared error regression for each diffusion denoising step.

• Anomaly Detection Domain: The OCC loss penalizes the norm of residual features for normal patches, pulling them into a hypersphere:

$\mathbf{p}_t^{0}$6

The maximum-likelihood loss on a flow model

$\mathbf{p}_t^{0}$7

encodes the density in residual space, supplementing with a boundary-guided push-pull (BG-SPP) margin.

• Vision/Medical: The memory bank is constructed via a greedy k-center approach, preserving coverage with significant compression (Niu et al., 2023).

4. Empirical Results and Benchmark Performance

4.1. Autonomous Driving

On the NAVSIM v1 benchmark, ResAD (with camera and LiDAR input, ResNet-34 encoder) achieves a PDM Score (PDMS) of 88.6, outperforming strong alternatives such as Transfuser (PDMS 84.0), DiffusionDrive (88.1), and WoTE (88.3). On NAVSIM v2 (EPDMS metric), ResAD attains 85.5. These results are notable for requiring only two DDIM denoising steps at inference, attributed to the narrow dynamic range of normalized residuals (Zheng et al., 9 Oct 2025).

Model	PDMS (NAVSIM v1)
Base	84.9
+ TRM	86.3
+ PRNorm	87.2
Full ResAD	88.6

4.2. Anomaly Detection (Cross-Class)

When trained on MVTecAD and tested on VisA, BTAD, MVTec3D, BraTS, and ShanghaiTech, ResAD yields substantial improvements in cross-domain AUROC:

Method	2-shot Img-AUROC	2-shot Px-AUROC	4-shot Img-AUROC	4-shot Px-AUROC
RegAD (prior)	74.4	94.2	79.0	94.9
ResAD (Yao et al., 2024)	80.9	95.8	85.8	97.0

Enhancements are even more pronounced when using more expressive backbones (e.g., ImageBind). Superior generalization is maintained even on medical (BraTS) and video (ShanghaiTech) domains.

4.3. Fundus Imaging

On IDRiD (retinal, medical) pixel-level AUC is 0.907 (vs. PatchCore 0.845), image-level AUC is 0.941 (vs. fAnoGAN 0.780). Even with only 10% training data, ReSAD retains competitive performance (image-AUC 0.913), underscoring the efficiency and robustness conferred by its region+spatial residual descriptors (Niu et al., 2023).

5. Interpretability, Causality, and Robustness

A central claim underlying residual-based methods is enhanced causal focus, interpretability, and safety:

• Causal Reasoning (E2EAD): By forcing the model to predict why the vehicle must deviate from an inertial path—rather than the aggregate path itself—the network is incentivized to identify scene facts (such as traffic lights, rules, or obstacles) as explanatory variables and rely less on superficial or confounding cues (Zheng et al., 9 Oct 2025).
• Feature Invariance (Anomaly Detection): Subtracting the closest normal patch removes class identity from feature vectors, so normal and anomalous residuals are tightly clustered across classes (empirical t-SNE confirmation in (Yao et al., 2024)). The constraintor and normalizing-flow further sharpen separation in the residual domain, yielding improved OOD detection without per-class retraining.
• Robustness and Safety: Tight normalization enables rapid diffusion denoising (for prediction) and mitigates long-horizon overfitting, while explicit model perturbations (e.g., inertial reference perturbation) yield diverse, plausible futures. In anomaly localization, the combination of regional and spatial context suppresses false positives along salient but normal structures (Niu et al., 2023).

6. Variants, Extensions, and Domain Adaptations

While ResAD architectures share the residual theme, their implementation details are domain-specific:

• Trajectory Modeling: Conditional diffusion decoders, inertial counterfactuals, point-wise normalization, and two-step DDIM denoising in autonomous driving.
• Class-Generalizable AD: Residual feature learning, shallow constraintors, normalizing flows, and boundary-aware push-pull objectives for transfer across classes and domains.
• Region+Spatial Vision: Convolutional and self-attention fusions for fundus imaging, compressed memory banks, and pixel-level anomaly scoring.
• Further Adaptations: Related approaches (e.g., RESTAD (Ghorbani et al., 2024), which uses RBF similarity in latent space in time-series anomaly detection), deploy variations on residual correlation and normalization to achieve general robustness.

A plausible implication is that residual-based architectures are extensible across structured prediction, visual discrimination, and unsupervised detection, provided that a suitable choice of reference (physical, distributional, or local normality) is available.

7. Significance and Prospects

The ResAD paradigm reframes dense prediction, detection, and modeling tasks by identifying and normalizing deviation from expected or normal behavior, often through physics- or data-driven references. This supports:

• Causal focus and interpretability, as seen in end-to-end driving tasks.
• Scalable generalization, enabling direct transfer of detectors to previously unseen domains with minimal or no adaptation, particularly in industrial anomaly detection and medical imaging.
• Stable, accelerated optimization by rebalancing targets and suppressing the dominance of high-variance outliers.

The approach is domain-agnostic and amenable to composition with recent advances (e.g., stronger feature encoders, efficient denoisers, memory bank compressions), suggesting continued impact on robust deep learning systems across fields (Zheng et al., 9 Oct 2025, Yao et al., 2024, Niu et al., 2023).