---
title: Multi-Output Robust Conjugate GP
url: https://www.emergentmind.com/topics/multi-output-rcgp-mo-rcgp
type: topic
---

# Multi-Output Robust Conjugate GP

Multi-output RCGP (MO-RCGP) denotes the **Multi-Output Robust and Conjugate Gaussian Process**, a robust multi-output Gaussian-process regression framework that extends the robust and conjugate Gaussian process (RCGP) construction from the scalar-output setting to correlated vector-valued responses. It replaces the standard Gaussian likelihood update with a **weighted Fisher (score-matching) loss**, yielding a **generalised-Bayes pseudo-posterior** that remains Gaussian in closed form, jointly captures correlations across outputs, and is **provably robust** to anomalous observations whose effects would otherwise propagate through cross-output correlations in conventional MOGPs [2510.26401].

## 1. Definition, scope, and naming

MO-RCGP is formulated for supervised regression with multiple correlated outputs. The observed data are \(N\) input-output pairs, with \(x_i \in \mathbb{R}^d\) and \(y_i \in \mathbb{R}^T\), and the latent function is vector-valued:
\[
f:\mathbb{R}^d \to \mathbb{R}^T,\qquad f(x) = [f_1(x),\ldots,f_T(x)]^\top.
\]
The defining objective is to retain the principal structural advantage of standard multi-output Gaussian processes—namely, a coherent joint prior over outputs—while avoiding their well-known sensitivity to model misspecification and outliers [2510.26401].

The method is designed for settings in which contamination in one output channel can distort predictions in other channels because the outputs are coupled through the covariance operator. In the terminology of the source paper, MO-RCGP is **conjugate**, **jointly captures correlations across outputs**, and is **provably robust** [2510.26401].

A point of nomenclature is necessary. The acronym “MO-RCGP” is also used in a later exposition of the **Multi-Output Recursive Consensus Gaussian Process** associated with the “Consensus-based Recursive Multi-Output Gaussian Process” framework, which addresses distributed and streaming inference through recursive information-form updates and neighbor-to-neighbor consensus [2604.10146]. That usage concerns a distinct problem class—distributed large-scale sensing rather than robust generalised-Bayes inference—and should not be conflated with Multi-Output Robust and Conjugate Gaussian Processes.

## 2. Multi-output GP formulation and the standard MOGP baseline

MO-RCGP begins from the standard multi-output GP construction. A prior is placed on \(f\) as
\[
f \sim GP(m,\mathcal{K}),
\]
where \(m:\mathbb{R}^d \to \mathbb{R}^T\) and \(\mathcal{K}:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}^{T \times T}\). The output-wise covariance is
\[
\operatorname{Cov}[f_t(x), f_{t'}(x')] = [\mathcal{K}(x,x')]_{t,t'} =: k_{t,t'}(x,x').
\]
A common kernel family is the **intrinsic coregionalisation model (ICM)**,
\[
\mathcal{K}(x,x') = B \cdot \kappa(x,x'),
\]
with \(B \in \mathbb{R}^{T \times T}\) positive semidefinite and \(\kappa\) a scalar kernel [2510.26401].

With noisy observations
\[
y_i = f(x_i) + \epsilon_i,\qquad \epsilon_i \sim N(0,\Sigma)\ \text{independently},
\]
and
\[
\Sigma = \operatorname{diag}(\sigma_1^2,\ldots,\sigma_T^2)\otimes I_N \in \mathbb{R}^{NT \times NT},
\]
the stacked latent vector \(f_{\mathrm{vec}}\) and data vector \(y_{\mathrm{vec}}\) are of dimension \(NT\). Over the training inputs \(X\), the prior is
\[
p(f_{\mathrm{vec}}\mid X) = N(f_{\mathrm{vec}}; m_{\mathrm{vec}}, K),
\]
where \(K = [\mathcal{K}(x_i,x_j)]_{i,j=1\ldots N}\) [2510.26401].

The standard MOGP posterior predictive at a test input \(x_*\) is Gaussian:
\[
p(f_* \mid x_*,Y,X) = N(f_*; \mu_*^{MOGP}, \Sigma_*^{MOGP}),
\]
with
\[
\mu_*^{MOGP} = m_* + k_*^\top (K+\Sigma)^{-1}(y_{\mathrm{vec}}-m_{\mathrm{vec}}),
\]
\[
\Sigma_*^{MOGP} = \mathcal{K}(x_*,x_*) - k_*^\top (K+\Sigma)^{-1}k_*.
\]
This baseline is fully conjugate, but the data explicitly note that standard MOGP is **highly sensitive to outliers**, and contamination in one channel can propagate to others through the cross-output covariance structure [2510.26401].

This sensitivity is not merely a pathological corner case. Related multi-output GP work on gap filling under the linear model of coregionalization shows the practical importance of nonzero cross-output covariance for information transfer across outputs, for example between LAI and fAPAR time series [2012.05912]. A plausible implication is that the same cross-output pathway that enables beneficial transfer in clean settings also provides a mechanism by which anomalous observations can contaminate coupled predictions.

## 3. Robustification via weighted Fisher divergence

The central methodological move in MO-RCGP is to replace the Gaussian likelihood update with a **weighted score-matching loss**. Let
\[
W(x_i,y_i) = \operatorname{diag}(w_1(x_i,y_i),\ldots,w_T(x_i,y_i))
\]
be a positive-definite \(T \times T\) weight matrix, typically diagonal. The method defines a generalised-Bayes pseudo-posterior
\[
p_W(f_{\mathrm{vec}}\mid Y,X) \propto p(f_{\mathrm{vec}}\mid X)\cdot \exp\!\bigl(-N\cdot \mathcal{L}_W(f_{\mathrm{vec}};Y,X)\bigr),
\]
where \(\mathcal{L}_W\) is the empirical weighted Fisher loss [2510.26401].

The weighted Fisher loss is
\[
\mathcal{L}_W = \frac{1}{N}\sum_{i=1}^N \|W_i^\top s_{\mathrm{model}}(x_i,y_i)\|^2 + 2\cdot \nabla_y \cdot [W_iW_i^\top s_{\mathrm{model}}(x_i,y_i)],
\]
with
\[
s_{\mathrm{model}}(x,y) = \nabla_y \log p(y\mid f(x))
\]
the score of the assumed Gaussian noise model \(N(y;f(x),\Sigma)\). The source derives
\[
s_{\mathrm{model}}(x_i,y_i)=\Sigma^{-1}(f(x_i)-y_i),
\]
and states that \(\mathcal{L}_W\) is a quadratic form in \(f_{\mathrm{vec}}\) [2510.26401].

The practical consequence is that the robustification can be expressed as a replacement of the standard noise structure by a weighted matrix term \(\Sigma J_W\), together with an adjusted mean \(m_{W,\mathrm{vec}}\). The required quantities are
\[
J_W = \frac{1}{2}\Sigma W^{-2},
\qquad
m_{W,\mathrm{vec}} = m_{\mathrm{vec}} + \Sigma \nabla_y \cdot \log(W^2).
\]

This construction is the source of the method’s dual characterization as **robust** and **conjugate**. It is robust because the weighting attenuates the influence of aberrant residuals, and conjugate because the resulting pseudo-posterior remains Gaussian in closed form [2510.26401].

## 4. Posterior, predictive distribution, and hyperparameter learning

The closed-form posterior is given in Proposition 3.1 of the source. The pseudo-posterior remains Gaussian:
\[
p_W(f_{\mathrm{vec}}\mid Y,X) = N(f_{\mathrm{vec}}; \mu^{MORCGP}, \Sigma^{MORCGP}),
\]
with
\[
\Sigma^{MORCGP} = [K^{-1} + (\Sigma \cdot J_W)^{-1}]^{-1}
= K(K+\Sigma J_W)^{-1}\Sigma J_W,
\]
and
\[
\mu^{MORCGP} = m_{\mathrm{vec}} + K(K+\Sigma J_W)^{-1}(y_{\mathrm{vec}}-m_{W,\mathrm{vec}}).
\]
The predictive distribution at \(x_*\) is likewise Gaussian:
\[
p_W(f_*\mid x_*,Y,X) = N(f_*; \mu_*^{MORCGP}, \Sigma_*^{MORCGP}),
\]
where
\[
\mu_*^{MORCGP} = m_* + k_*^\top (K+\Sigma J_W)^{-1}(y_{\mathrm{vec}}-m_{W,\mathrm{vec}}),
\]
\[
\Sigma_*^{MORCGP} = \mathcal{K}(x_*,x_*) - k_*^\top (K+\Sigma J_W)^{-1}k_*.
\]
These expressions reduce exactly to standard MOGP when
\[
W_i = (\sigma_t/\sqrt{2})I
\]
for all \(i,t\) [2510.26401].

A critical methodological point is that this posterior is **not a true likelihood Bayes update**. Consequently, the usual marginal likelihood is not the appropriate objective for hyperparameter selection. Instead, MO-RCGP uses a **weighted leave-one-out cross-validation objective (w-LOO-CV)**:
\[
\Phi_w(\Sigma,\theta)
=
\sum_{i=1}^N\sum_{t=1}^T
(w_{i,t}/\beta_t)^2
\cdot
\log p_W(y_{i,t}\mid Y_{-(i,t)},\Sigma,\theta),
\]
where \(\beta_t = \sigma_t/\sqrt{2}\) rescales the weights. The leave-one-out predictive is available in closed-form Gaussian form and depends only on \((K+\Sigma J_W)^{-1}\), enabling gradient-based optimization such as **L-BFGS** [2510.26401].

## 5. Robustness guarantees and computational profile

The method’s theoretical positioning is most clearly understood by comparing it with neighboring model classes.

| Model | Statistical property | Computational/inference note |
|---|---|---|
| Standard MOGP | Fully conjugate but highly sensitive to outliers | Same \(O(N^3T^3)\) scaling |
| Student-\(t\) MOGP | Robust but breaks conjugacy | Requires variational inference and \(\sim 6\times\) the compute |
| RCGP | Robust and conjugate, but only for \(T=1\) | Scalar-output setting |
| MO-RCGP | Robust, conjugate, and multi-output | Same \(O(N^3T^3)\) scaling as standard MOGP |

The source paper states three computational facts. First, forming \(K\) or \(K+\Sigma J_W\) and factoring it costs \(O(N^3T^3)\). Second, all posterior and predictive formulae reuse this factorization. Third, **no additional approximate inference (e.g. variational) is required**. Hence MO-RCGP scales exactly like standard MOGP in \(N\) and \(T\) [2510.26401].

Its robustness claim is stronger than empirical resistance to outliers. The paper states that MO-RCGP has **provable robustness**: the posterior influence function in any channel is bounded even as a single observation’s error tends to infinity. This is identified as Proposition 3.2 in the source [2510.26401].

A common misconception is that robustness in multi-output GP models necessarily requires heavy-tailed observation models or mixture likelihoods. MO-RCGP is explicitly positioned against that view. The paper’s summary states that, unlike heavy-tailed or mixture likelihoods, it **retains the same computational cost as standard MOGP** while remaining fully conjugate [2510.26401].

## 6. Empirical behavior, application domains, and related developments

The empirical evaluation spans synthetic and real settings in which anomalous observations occur in one or more output channels. In a synthetic multi-task imputation experiment, the setup used \(T=2\) outputs, ICM with
\[
B=\begin{bmatrix}2 & 1.25\\ 1.25 & 1\end{bmatrix},
\]
\(\ell=1\), \(N=120\), 2.5% outliers in the second channel, and missing values in the first on \([0.3,0.7]\). The reported qualitative outcome is that **MOGP is distorted by outliers; MO-RCGP remains robust** [2510.26401].

In a second synthetic comparison, the data specify \(T=3\), \(N=100\), a coregionalization matrix with large off-diagonals, \(\ell=1\), noise standard deviation \(0.1\), and 10% outliers in channel 1 uniform on \([2,3]\) or clean. Metrics were **RMSE** and **NLPD** over 20 seeds. The reported conclusion is that **MOGP degrades under outliers; MO-RCGP (with proposed weight) matches or outperforms t-MOGP while costing \(\sim 1/6\) the compute** [2510.26401].

On the **Energy Efficiency (UCI)** dataset, with \(N=768\), \(d=8\), and \(T=2\) outputs corresponding to heating and cooling loads, the evaluation considered outlier scenarios described as none, uniform, asymmetric, and focused 10% in heating load. Metrics were RMSE, NLPD, and wall-clock seconds. The reported pattern is that **MOGP degrades drastically under outliers**, whereas **MO-RCGP matches MOGP when no outliers, outperforms MOGP and t-MOGP when outliers are present, and runs in \(\approx 50\) s versus \(\approx 330\) s for t-MOGP** [2510.26401].

Two domain-specific case studies emphasize the role of cross-output robustness. In **cancer dose-response**, using Navitoclax viability data for 2 cell lines with \(N=21\) dose points in \(\log_{10}\,\mu\mathrm{M}\), one red outlier causes MOGP to over-fit and inflate uncertainty, while MO-RCGP **downweights that point and yields sharper, unbiased curves**. In the **financial TBA market**, with \(T=3\) MBS coupons (5%, 5.5%, 6%) and \(N=139\) trades over 2 days at 1 minute resolution, one 5.5% outlier at 4 pm distorts MOGP’s fit for 5.5% and propagates error to 6%, whereas **MO-RCGP remains stable in all three channels** [2510.26401].

The broader multi-output GP literature provides context for why these results matter. Multi-output models under the linear model of coregionalization have been shown to exploit cross-output covariance to fill severe gaps in biophysical time series, substantially outperforming single-output baselines when LAI and fAPAR overlap in time [2012.05912]. This suggests that MO-RCGP inherits the principal structural benefit of MOGPs—cross-domain information transfer—while modifying the inferential update so that sparse contamination does not dominate coupled predictions.

Finally, the acronym overlap with the **Multi-Output Recursive Consensus Gaussian Process** should be kept explicit. The consensus-based framework uses shared basis vectors, information-form recursion, and neighbor-to-neighbor averaging of information parameters for parallel, fully distributed learning with bounded per-step computation in multi-agent sensing [2604.10146]. By contrast, Multi-Output Robust and Conjugate Gaussian Processes target robustness to outliers and model misspecification in centralized multi-output regression. The two methods share the multi-output GP substrate but differ in objective, inference mechanism, and deployment regime.

Source: https://www.emergentmind.com/topics/multi-output-rcgp-mo-rcgp