Residual Distribution Predictive Systems

• Residual Distribution Predictive Systems are predictive methods that construct distributions from regression residuals, ensuring calibrated marginal coverage.
• They integrate with point regression models by translating training residuals to form lower and upper empirical prediction bounds.
• RDPS expand the scope of predictive inference by supporting complex models without strict monotonicity constraints, adapting uncertainty to local data.

Residual Distribution Predictive Systems (RDPS) are predictive systems for real-valued outcomes that construct sets of predictive distributions with out-of-sample calibration guarantees by leveraging residual-based forecasting procedures rather than conformity measures as the primary organizing device. They were introduced as an approach that, in the split conformal setting, nests conformal predictive systems built from a popular class of residual-based conformity measures, while in the full conformal setting they differ in a way that removes the need for fairly stringent monotonicity requirements for well-definedness. The resulting framework is intended to work alongside point-valued regression methods, including flexible regression and machine-learning models, while retaining marginal calibration guarantees under exchangeability (Allen et al., 30 Oct 2025).

1. Predictive-system formulation

RDPS are situated within the general theory of predictive systems, where a predictive system $\Pi$ is represented as a set of pairs $(y,\tau)$ with $y \in \mathbb{R}$ and $\tau$ lying between lower and upper distribution functions, $\Pi_\ell(y) \le \tau \le \Pi_u(y)$. In this setting, calibration is probabilistic calibration: for any $\alpha \in (0,1)$, the predictive CDF evaluated at the realized outcome satisfies

$$\mathbb{P}(G(Y) \le \alpha) \le \alpha \le \mathbb{P}(G(Y-) < \alpha).$$

The significance of this formulation is that the guarantee is distributional rather than merely interval-specific: the set of predictive distributions contains a forecast distribution whose prediction intervals exhibit the correct marginal coverage at all levels (Allen et al., 30 Oct 2025).

The basic RDPS forecasting device begins with training data $(x_i,y_i)_{i=1}^n$ and a point regression predictor $\hat y_x$. Writing $\hat\varepsilon_i = y_i - \hat y_i$ for the residual associated with observation $(y,\tau)$0, the residual-distribution forecast at covariate value $(y,\tau)$1 is

$(y,\tau)$2

This construction may be read as an empirical distribution obtained by translating the training residuals to the prediction site through the fitted point predictor. A plausible implication is that RDPS separate the regression problem from the predictive-distribution construction more explicitly than approaches centered on conformity scores.

2. Construction of the lower and upper predictive bounds

For a new covariate $(y,\tau)$3, RDPS convert the residual-distribution forecast into a predictive system by considering empirical distributions augmented with hypothetical outcomes. If $(y,\tau)$4 denotes the empirical distribution augmented with $(y,\tau)$5, then the lower and upper bounds of the predictive system are defined by

$(y,\tau)$6

$(y,\tau)$7

These envelopes encode the ambiguity induced by augmenting the sample with the test object and an arbitrary candidate label. In effect, RDPS inherit the predictive-system logic of conformal methods, but express it through residual-distribution forecasts rather than starting from a conformity measure (Allen et al., 30 Oct 2025).

The paper also allows a generalized residual transformation. In the split setting, if the conformity measure has the form

$(y,\tau)$8

with $(y,\tau)$9 strictly increasing, then the associated generalized residual-distribution forecast can be written as

$y \in \mathbb{R}$0

This form is central to the equivalence result with split conformal predictive systems.

3. Relation to conformal predictive systems

The most important structural fact about RDPS is that they are not simply a competitor to conformal predictive systems (CPS). In the split conformal setting, RDPS are equivalent to CPS when the conformity measure is residual-based in the sense just described. The paper states that any conformity measure of the form $y \in \mathbb{R}$1, with $y \in \mathbb{R}$2 strictly increasing, yields a conformal predictive system that is equivalent to a generalized RDPS with residual transformation. In this regime, RDPS provide an alternative perspective on the classical residual-based conformal construction rather than a different inferential object (Allen et al., 30 Oct 2025).

This equivalence clarifies a common misconception. RDPS do not supplant conformal predictive systems in the split case; rather, they reinterpret an important subclass of them. The paper also states that both approaches yield predictive systems with thickness $y \in \mathbb{R}$3 in the split setting. Accordingly, the split-conformal comparison is best understood as one of representation and implementation, not one of calibration strength.

4. Divergence from full conformal methods

The distinction becomes substantive in the full conformal setting. There, CPS require the conformity measure to satisfy a monotonicity condition for the predictive-system bounds to be valid and well-defined. RDPS do not require this condition. This difference is the principal theoretical advantage emphasized by the paper: RDPS can be implemented alongside any point-valued regression method to yield predictive systems with out-of-sample calibration guarantees, whereas full conformal predictive systems may exclude regression procedures that do not induce suitably monotone conformity scores (Allen et al., 30 Oct 2025).

Aspect	CPS	RDPS
Split conformal setting	Equivalent to RDPS for residual-based conformity measures	Equivalent to CPS in the residual-based split setting
Full conformal setting	Requires monotonicity condition for well-definedness	Does not require monotonicity condition
Thickness behavior	Fixed	Varies with covariates and model choice

The thickness behavior is especially consequential. The paper reports that CPS thickness is fixed, whereas RDPS thickness varies with covariates and model choice. It further notes that this variation reflects local epistemic uncertainty, but can sometimes be prohibitively wide for flexible models unless robust regression is adopted. This suggests that RDPS trade structural flexibility for a more model-sensitive uncertainty geometry: their predictive sets can adapt to the regression method and the local covariate configuration, but this adaptivity can enlarge the predictive-system thickness in practice.

5. Empirical assessment on simulated regression problems

The empirical study uses two simulated regression settings. The first is linear: $y \in \mathbb{R}$4 The second is nonlinear and heteroscedastic: $y \in \mathbb{R}$5 The compared methods include RDPS using OLS regression, including “deleted” RDPS for robustness, RDPS using Kernel Ridge Regression, LSPM, and KRRPM. Evaluation is based on interval coverage, interval width, and average interval score (Allen et al., 30 Oct 2025).

Across these experiments, all methods yield valid, often slightly conservative, coverage. With OLS, interval widths are similar for CPS and RDPS. With Kernel Ridge Regression, RDPS intervals may be wider because of greater epistemic uncertainty. Interval scores are reported as comparable between CPS and RDPS, and more flexible regressors yield narrower intervals in nonlinear cases but with increased thickness for RDPS. The overall empirical conclusion is not that RDPS dominate CPS uniformly, but that they perform competitively while opening a wider implementation space for alternative regression methods.

6. Scope, applicability, and terminological boundaries

The principal practical implication of RDPS is that predictive inference with out-of-sample calibration guarantees can be attached to a much broader class of regression procedures than is readily available in full conformal predictive systems. The paper explicitly highlights random forests and neural networks as examples of the expanded applicability. In high-data settings, split-conformal procedures remain efficient and reliable, and in that regime RDPS are equivalent to CPS for standard residual-based constructions. In low-data or complex-model settings, RDPS offer greater flexibility because they avoid the monotonicity restriction that full conformal predictive systems impose (Allen et al., 30 Oct 2025).

A second misconception concerns terminology. The phrase “residual distribution” already has an established, unrelated meaning in numerical analysis, where “residual distribution schemes” or “fluctuation splitting schemes” denote discretization methods for hyperbolic balance laws and related PDEs (Abgrall et al., 2021). RDPS belong instead to the literature on predictive systems, distributional forecasting, and calibrated uncertainty quantification. The shared phrase is historical coincidence rather than methodological continuity.

Taken together, RDPS can be understood as a residual-based route to predictive systems that unifies part of split conformal predictive inference, departs from full conformal methods where monotonicity becomes restrictive, and exposes a broader design space for coupling calibrated predictive distributions with modern regression machinery.