Multi-Output Robust Conjugate GP

• The paper introduces MO-RCGP, a framework that replaces standard Gaussian likelihood updates with a weighted Fisher loss to achieve robust, fully conjugate multi-output predictions.
• It employs a closed-form Gaussian pseudo-posterior that jointly captures correlations across outputs, mitigating contamination effects from outlier channels.
• The method scales similarly to standard MOGPs yet outperforms them under data anomalies, offering efficient hyperparameter learning via weighted leave-one-out CV.

Multi-output RCGP (MO-RCGP) denotes the Multi-Output Robust and Conjugate Gaussian Process, a robust multi-output Gaussian-process regression framework that extends the robust and conjugate Gaussian process (RCGP) construction from the scalar-output setting to correlated vector-valued responses. It replaces the standard Gaussian likelihood update with a weighted Fisher (score-matching) loss, yielding a generalised-Bayes pseudo-posterior that remains Gaussian in closed form, jointly captures correlations across outputs, and is provably robust to anomalous observations whose effects would otherwise propagate through cross-output correlations in conventional MOGPs (Rooijakkers et al., 30 Oct 2025).

1. Definition, scope, and naming

MO-RCGP is formulated for supervised regression with multiple correlated outputs. The observed data are $N$ input-output pairs, with $x_i \in \mathbb{R}^d$ and $y_i \in \mathbb{R}^T$, and the latent function is vector-valued: $$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$ The defining objective is to retain the principal structural advantage of standard multi-output Gaussian processes—namely, a coherent joint prior over outputs—while avoiding their well-known sensitivity to model misspecification and outliers (Rooijakkers et al., 30 Oct 2025).

The method is designed for settings in which contamination in one output channel can distort predictions in other channels because the outputs are coupled through the covariance operator. In the terminology of the source paper, MO-RCGP is conjugate, jointly captures correlations across outputs, and is provably robust (Rooijakkers et al., 30 Oct 2025).

A point of nomenclature is necessary. The acronym “MO-RCGP” is also used in a later exposition of the Multi-Output Recursive Consensus Gaussian Process associated with the “Consensus-based Recursive Multi-Output Gaussian Process” framework, which addresses distributed and streaming inference through recursive information-form updates and neighbor-to-neighbor consensus (Rao et al., 11 Apr 2026). That usage concerns a distinct problem class—distributed large-scale sensing rather than robust generalised-Bayes inference—and should not be conflated with Multi-Output Robust and Conjugate Gaussian Processes.

2. Multi-output GP formulation and the standard MOGP baseline

MO-RCGP begins from the standard multi-output GP construction. A prior is placed on $f$ as

$f \sim GP(m,\mathcal{K}),$

where $m:\mathbb{R}^d \to \mathbb{R}^T$ and $$\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}$$. The output-wise covariance is

$$\operatorname{Cov}[f_t(x), f_{t'}(x')] = [\mathcal{K}(x,x')]_{t,t'} =: k_{t,t'}(x,x').$$

A common kernel family is the intrinsic coregionalisation model (ICM),

$\mathcal{K}(x,x') = B \cdot \kappa(x,x'),$

with $x_i \in \mathbb{R}^d$0 positive semidefinite and $x_i \in \mathbb{R}^d$1 a scalar kernel (Rooijakkers et al., 30 Oct 2025).

With noisy observations

$x_i \in \mathbb{R}^d$2

and

$x_i \in \mathbb{R}^d$3

the stacked latent vector $x_i \in \mathbb{R}^d$4 and data vector $x_i \in \mathbb{R}^d$5 are of dimension $x_i \in \mathbb{R}^d$6. Over the training inputs $x_i \in \mathbb{R}^d$7, the prior is

$x_i \in \mathbb{R}^d$8

where $x_i \in \mathbb{R}^d$9 (Rooijakkers et al., 30 Oct 2025).

The standard MOGP posterior predictive at a test input $y_i \in \mathbb{R}^T$0 is Gaussian: $y_i \in \mathbb{R}^T$1 with

$y_i \in \mathbb{R}^T$2

$y_i \in \mathbb{R}^T$3

This baseline is fully conjugate, but the data explicitly note that standard MOGP is highly sensitive to outliers, and contamination in one channel can propagate to others through the cross-output covariance structure (Rooijakkers et al., 30 Oct 2025).

This sensitivity is not merely a pathological corner case. Related multi-output GP work on gap filling under the linear model of coregionalization shows the practical importance of nonzero cross-output covariance for information transfer across outputs, for example between LAI and fAPAR time series (Mateo-Sanchis et al., 2020). A plausible implication is that the same cross-output pathway that enables beneficial transfer in clean settings also provides a mechanism by which anomalous observations can contaminate coupled predictions.

3. Robustification via weighted Fisher divergence

The central methodological move in MO-RCGP is to replace the Gaussian likelihood update with a weighted score-matching loss. Let

$y_i \in \mathbb{R}^T$4

be a positive-definite $y_i \in \mathbb{R}^T$5 weight matrix, typically diagonal. The method defines a generalised-Bayes pseudo-posterior

$y_i \in \mathbb{R}^T$6

where $y_i \in \mathbb{R}^T$7 is the empirical weighted Fisher loss (Rooijakkers et al., 30 Oct 2025).

The weighted Fisher loss is

$y_i \in \mathbb{R}^T$8

with

$y_i \in \mathbb{R}^T$9

the score of the assumed Gaussian noise model $$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$0. The source derives

$$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$1

and states that $$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$2 is a quadratic form in $$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$3 (Rooijakkers et al., 30 Oct 2025).

The practical consequence is that the robustification can be expressed as a replacement of the standard noise structure by a weighted matrix term $$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$4, together with an adjusted mean $$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$5. The required quantities are

$$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$6

This construction is the source of the method’s dual characterization as robust and conjugate. It is robust because the weighting attenuates the influence of aberrant residuals, and conjugate because the resulting pseudo-posterior remains Gaussian in closed form (Rooijakkers et al., 30 Oct 2025).

4. Posterior, predictive distribution, and hyperparameter learning

The closed-form posterior is given in Proposition 3.1 of the source. The pseudo-posterior remains Gaussian: $$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$7 with

$$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$8

and

$$f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.$$9

The predictive distribution at $f$0 is likewise Gaussian: $f$1 where

$f$2

$f$3

These expressions reduce exactly to standard MOGP when

$f$4

for all $f$5 (Rooijakkers et al., 30 Oct 2025).

A critical methodological point is that this posterior is not a true likelihood Bayes update. Consequently, the usual marginal likelihood is not the appropriate objective for hyperparameter selection. Instead, MO-RCGP uses a weighted leave-one-out cross-validation objective (w-LOO-CV): $f$6 where $f$7 rescales the weights. The leave-one-out predictive is available in closed-form Gaussian form and depends only on $f$8, enabling gradient-based optimization such as L-BFGS (Rooijakkers et al., 30 Oct 2025).

5. Robustness guarantees and computational profile

The method’s theoretical positioning is most clearly understood by comparing it with neighboring model classes.

Model	Statistical property	Computational/inference note
Standard MOGP	Fully conjugate but highly sensitive to outliers	Same $f$9 scaling
Student-$f \sim GP(m,\mathcal{K}),$0 MOGP	Robust but breaks conjugacy	Requires variational inference and $f \sim GP(m,\mathcal{K}),$1 the compute
RCGP	Robust and conjugate, but only for $f \sim GP(m,\mathcal{K}),$2	Scalar-output setting
MO-RCGP	Robust, conjugate, and multi-output	Same $f \sim GP(m,\mathcal{K}),$3 scaling as standard MOGP

The source paper states three computational facts. First, forming $f \sim GP(m,\mathcal{K}),$4 or $f \sim GP(m,\mathcal{K}),$5 and factoring it costs $f \sim GP(m,\mathcal{K}),$6. Second, all posterior and predictive formulae reuse this factorization. Third, no additional approximate inference (e.g. variational) is required. Hence MO-RCGP scales exactly like standard MOGP in $f \sim GP(m,\mathcal{K}),$7 and $f \sim GP(m,\mathcal{K}),$8 (Rooijakkers et al., 30 Oct 2025).

Its robustness claim is stronger than empirical resistance to outliers. The paper states that MO-RCGP has provable robustness: the posterior influence function in any channel is bounded even as a single observation’s error tends to infinity. This is identified as Proposition 3.2 in the source (Rooijakkers et al., 30 Oct 2025).

A common misconception is that robustness in multi-output GP models necessarily requires heavy-tailed observation models or mixture likelihoods. MO-RCGP is explicitly positioned against that view. The paper’s summary states that, unlike heavy-tailed or mixture likelihoods, it retains the same computational cost as standard MOGP while remaining fully conjugate (Rooijakkers et al., 30 Oct 2025).

6. Empirical behavior, application domains, and related developments

The empirical evaluation spans synthetic and real settings in which anomalous observations occur in one or more output channels. In a synthetic multi-task imputation experiment, the setup used $f \sim GP(m,\mathcal{K}),$9 outputs, ICM with

$m:\mathbb{R}^d \to \mathbb{R}^T$0

$m:\mathbb{R}^d \to \mathbb{R}^T$1, $m:\mathbb{R}^d \to \mathbb{R}^T$2, 2.5% outliers in the second channel, and missing values in the first on $m:\mathbb{R}^d \to \mathbb{R}^T$3. The reported qualitative outcome is that MOGP is distorted by outliers; MO-RCGP remains robust (Rooijakkers et al., 30 Oct 2025).

In a second synthetic comparison, the data specify $m:\mathbb{R}^d \to \mathbb{R}^T$4, $m:\mathbb{R}^d \to \mathbb{R}^T$5, a coregionalization matrix with large off-diagonals, $m:\mathbb{R}^d \to \mathbb{R}^T$6, noise standard deviation $m:\mathbb{R}^d \to \mathbb{R}^T$7, and 10% outliers in channel 1 uniform on $m:\mathbb{R}^d \to \mathbb{R}^T$8 or clean. Metrics were RMSE and NLPD over 20 seeds. The reported conclusion is that MOGP degrades under outliers; MO-RCGP (with proposed weight) matches or outperforms t-MOGP while costing $m:\mathbb{R}^d \to \mathbb{R}^T$9 the compute (Rooijakkers et al., 30 Oct 2025).

On the Energy Efficiency (UCI) dataset, with $$\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}$$0, $$\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}$$1, and $$\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}$$2 outputs corresponding to heating and cooling loads, the evaluation considered outlier scenarios described as none, uniform, asymmetric, and focused 10% in heating load. Metrics were RMSE, NLPD, and wall-clock seconds. The reported pattern is that MOGP degrades drastically under outliers, whereas MO-RCGP matches MOGP when no outliers, outperforms MOGP and t-MOGP when outliers are present, and runs in $$\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}$$3 s versus $$\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}$$4 s for t-MOGP (Rooijakkers et al., 30 Oct 2025).

Two domain-specific case studies emphasize the role of cross-output robustness. In cancer dose-response, using Navitoclax viability data for 2 cell lines with $$\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}$$5 dose points in $$\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}$$6, one red outlier causes MOGP to over-fit and inflate uncertainty, while MO-RCGP downweights that point and yields sharper, unbiased curves. In the financial TBA market, with $$\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}$$7 MBS coupons (5%, 5.5%, 6%) and $$\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}$$8 trades over 2 days at 1 minute resolution, one 5.5% outlier at 4 pm distorts MOGP’s fit for 5.5% and propagates error to 6%, whereas MO-RCGP remains stable in all three channels (Rooijakkers et al., 30 Oct 2025).

The broader multi-output GP literature provides context for why these results matter. Multi-output models under the linear model of coregionalization have been shown to exploit cross-output covariance to fill severe gaps in biophysical time series, substantially outperforming single-output baselines when LAI and fAPAR overlap in time (Mateo-Sanchis et al., 2020). This suggests that MO-RCGP inherits the principal structural benefit of MOGPs—cross-domain information transfer—while modifying the inferential update so that sparse contamination does not dominate coupled predictions.

Finally, the acronym overlap with the Multi-Output Recursive Consensus Gaussian Process should be kept explicit. The consensus-based framework uses shared basis vectors, information-form recursion, and neighbor-to-neighbor averaging of information parameters for parallel, fully distributed learning with bounded per-step computation in multi-agent sensing (Rao et al., 11 Apr 2026). By contrast, Multi-Output Robust and Conjugate Gaussian Processes target robustness to outliers and model misspecification in centralized multi-output regression. The two methods share the multi-output GP substrate but differ in objective, inference mechanism, and deployment regime.