Scaled Residuals

• Scaled residuals are a diagnostic tool that transforms model discrepancies using methods like the probability integral transform, yielding reference distributions (e.g., N(0,1) or Uniform) under correct specification.
• They generalize classical residuals to various data types—including continuous, discrete, semicontinuous, and high-dimensional models—by employing techniques such as randomization and double-PIT adjustments.
• By calibrating for the model’s predictive distribution, scaled residuals enhance the detection of model mis-specification and outliers and support the construction of formally calibrated test statistics.

Scaled residuals are a class of residual diagnostics designed to standardize model departures such that, under correct model specification, their distribution is known and interpretable—most often uniform or normal. These residuals generalize beyond classical standardized residuals to encompass continuous, discrete, censored, semicontinuous, high-dimensional, latent-variable (state-space), and point process models. Scaled residuals improve model diagnostics and inference by calibrating for the model's predictive distribution, enabling robust detection of lack-of-fit, outlier assessment, and the construction of formal test statistics under nominal Type I error. The formalism, computational procedures, and theoretical properties of scaled residuals have been developed in depth across recent literature, notably as percentile-based residuals, probability-scale residuals, double probability integral transform residuals, and rescaled statistics for different modeling domains.

1. Formal Definitions and Unifying Principles

The central unifying principle of scaled residuals is the probability integral transform (PIT) and its extensions. For an observation $y_i$ with covariates $x_i$ and fitted conditional CDF $\hat F_i(y)$, the canonical construction is:

• Percentile-based ("quantile") residual: $u_i = \hat F_i(y_i)$; $r_i = \Phi^{-1}(u_i)$, where $\Phi$ is the standard normal CDF (Bérubé et al., 2019).
• Randomization for discreteness: If $\hat F_i$ has jumps (discrete or mixed outcome), use $$U_i \sim \text{Uniform}(\hat F_i(y_i^-),\hat F_i(y_i))$$, then $r_i = \Phi^{-1}(U_i)$, or apply half-correction as in Dunn & Smyth (1996).

For uniformly distributed $u_i$, the transform $x_i$0 yields standard normal residuals under the true model. The same logic underlies the probability-scale residual (PSR), defined by the expectation $x_i$1, which for continuous models reduces to $x_i$2 and is uniform on $x_i$3 under correct specification (Shepherd et al., 2018).

For semicontinuous data (point mass at zero), the "scaled" residual is computed via an empirical correction (uniformization of the PIT) as $x_i$4, aligning the residual distribution with $x_i$5 (Yang, 2024). For discrete regression, double probability-integral transform constructions are necessary to decorrelate residuals from covariates (Yang, 2023).

In high-dimensional regression, scaled residuals refer to rescaled ordinary or Lasso residual vectors: $x_i$6 (OLS) or $x_i$7 (Lasso); these are pivotal and ancillary under the null (Shah et al., 2015). For space-time point processes, "rescaled residuals" refer to the transformation of event coordinates to achieve a homogeneous Poisson process under the model (Clements et al., 2012).

2. Distributional Properties and Calibration

The defining property of scaled residuals is that, under the true model, they follow a known reference distribution:

• For percentile-based or quantile residuals ($x_i$8): $x_i$9 if the fitted and true CDF match, and the data are continuous (Bérubé et al., 2019, Scudilio et al., 2017).
• For probability-scale residuals: $\hat F_i(y)$0 for continuous $\hat F_i(y)$1 (Shepherd et al., 2018).
• For semicontinuous models: $\hat F_i(y)$2 under the full model including the zero-mass (Yang, 2024).
• For space-time rescaled residuals: transformed points should constitute a unit-rate Poisson process (Clements et al., 2012).
• In the discrete setting, after double PIT and marginalization, the residuals are exactly standard normal (Yang, 2023).

If the model is mis-specified, the scaled residuals systematically depart from the reference distribution, providing diagnostic power.

3. Computation and Model-Specific Construction

The following summarizes major computational frameworks for scaled residuals:

Model Class	Residual formula(s)	Reference
General regression (continuous/discrete)	$\hat F_i(y)$3 (possibly randomized/half-corrected)	(Bérubé et al., 2019)
Semicontinuous models (e.g., Tobit, Tweedie)	$\hat F_i(y)$4 where $\hat F_i(y)$5 is empirically estimated	(Yang, 2024)
GLMs (Gamma, IG)	Adjusted quantile residual $\hat F_i(y)$6	(Scudilio et al., 2017)
State-space/MARSS models	$\hat F_i(y)$7 and similar for $\hat F_i(y)$8 (see Harvey–Koopman recursions)	(Holmes, 2014)
Point process (space-time)	Transform event coordinates via cumulative intensity integrals (rescaling theorem)	(Clements et al., 2012)
Discrete regression (double PIT)	$\hat F_i(y)$9, with $u_i = \hat F_i(y_i)$0 randomized PIT, $u_i = \hat F_i(y_i)$1 is leave-one-out empirical	(Yang, 2023)
High-dimensional regression	$u_i = \hat F_i(y_i)$2 (OLS); $u_i = \hat F_i(y_i)$3 as above (Lasso)	(Shah et al., 2015)

Closed-form CDFs are used when available; otherwise, empirical or simulation-based CDF estimation, randomized adjustment for ties, and empirical marginalization are applied. For complex models (e.g., mixed-effects, Bayesian hierarchical), predictive CDFs are generated via MCMC or parametric bootstrap (Bérubé et al., 2019).

4. Comparative Advantages over Classical Residuals

Classical standardized and deviance residuals (e.g., $u_i = \hat F_i(y_i)$4) fail to achieve reference normality under non-Gaussian or discrete distributions, leading to miscalibrated Type I error and potentially misleading diagnostics (Bérubé et al., 2019, Scudilio et al., 2017, Yang, 2023). In contrast, scaled residuals:

• Provide exact or asymptotic reference distributions (Uniform or $u_i = \hat F_i(y_i)$5) regardless of response type when the model is correct.
• Exhibit improved power for outlier detection and model misspecification identification.
• Avoid the need for ad hoc calibration of critical values.
• Support robust diagnostic plotting (e.g., QQ-plots versus standard normal or uniform, histograms, and residual-by-covariate plots) (Bérubé et al., 2019, Yang, 2023).
• Are applicable across continuous, discrete, and mixed-outcome models, including models with point masses, censored data, and time/space dependencies (Yang, 2024, Shepherd et al., 2018, Clements et al., 2012).

In simulation and case study benchmarks, scaled residuals consistently outperform classical residuals in normality, mean-centering, and calibration properties (Scudilio et al., 2017, Bérubé et al., 2019, Yang, 2023).

5. Application Domains and Illustrative Use Cases

Scaled residuals have broad applicability:

• Generalized linear models: Adjusted quantile residuals yield superior normality, ideal for GLM diagnostics under high dispersion or small $u_i = \hat F_i(y_i)$6 (Scudilio et al., 2017).
• Hierarchical and Bayesian models: Percentile-based residuals (and their randomization-aware variants) provide well-calibrated outlier detection and graphical fit assessment (Bérubé et al., 2019).
• Semicontinuous outcomes: Uniformization residuals validate Tobit, Tweedie, and two-part models; QQ-plot departures reveal omission of covariates or mispecification of the distributional tail (Yang, 2024).
• HIV/AIDS research: Probability-scale residuals enable diagnostics for continuous, ordinal, and censored outcomes; PSR-based partial rank correlations generalize Spearman's rho with adjustment for covariates (Shepherd et al., 2018).
• State-space models: Standardized model and innovation residuals permit outlier detection, cross-validation, and systematic diagnostic plotting following the Kalman smoothing recursion (Holmes, 2014).
• Point process analysis: Rescaled residuals diagnose spatial or space-time intensity model fit by transforming data to a homogeneous Poisson process; lack-of-fit manifests as clustering or inhibition in transformed coordinates (Clements et al., 2012).
• High-dimensional regression: Scaled residuals form the basis of model-specification tests (e.g., for heteroscedasticity, nonlinearity) that are exact or bootstrapped, and free from unknown nuisance parameters (Shah et al., 2015).
• Large-scale optimization: In iterative refinement for quadratic programming, residual scaling "zooms" violations up to solver-precision, ensuring systematic convergence of primal/dual KKT residuals (Weber et al., 2018).

6. Practical Recommendations and Diagnostic Strategies

Best practices for scaled residual construction and use:

• Always compute full predictive (conditional) distribution for each observation; employ posterior predictive or parametric bootstrap where analytic forms are unavailable (Bérubé et al., 2019).
• For discrete or censored outcomes, use randomized, half-correction, or double-PIT constructions to eliminate discretization artifacts (Yang, 2023, Bérubé et al., 2019).
• For semicontinuous or zero-inflated models, apply empirical uniformization procedures designed to account for point masses (Yang, 2024).
• Visual diagnosis: use QQ-plots versus Uniform or Normal reference, histograms, and "ordered curve" plots to detect mean-structure misspecification (Yang, 2023).
• Outlier flagging: values of $u_i = \hat F_i(y_i)$7 or $u_i = \hat F_i(y_i)$8 (or their uniform equivalents) indicate potentially extreme observations, with Type I error rates properly calibrated under the model (Bérubé et al., 2019).
• For time-series/state-space models, use Harvey-Koopman backward recursions for dynamic residual standardization (Holmes, 2014).
• In high-dimensional models, scaled residuals enable formal inference and diagnostic tests via simulation or parametric bootstrap (Shah et al., 2015).
• Address computational considerations—numerical integration, discretization and edge effects (in spatial/space-time models), and MCMC efficiency in Bayesian inference as needed (Clements et al., 2012).

Adoption of scaled residuals enhances reliability of diagnostic inference across contemporary statistical modeling regimes, superseding classical normalized residuals in robustness, interpretability, and theoretical guarantees.