Consensus-based Recursive Multi-Output Gaussian Process

Abstract: Multi-output Gaussian Processes provide principled uncertainty-aware learning of vector-valued fields but are difficult to deploy in large-scale, distributed, and streaming settings due to their computational and centralized nature. This paper proposes a Consensus-based Recursive Multi-Output Gaussian Process (CRMGP) framework that combines recursive inference on shared basis vectors with neighbour-to-neighbour information-consensus updates. The resulting method supports parallel, fully distributed learning with bounded per-step computation while preserving inter-output correlations and calibrated uncertainty. Experiments on synthetic wind fields and real LiDAR data demonstrate that CRMGP achieves competitive predictive performance and reliable uncertainty calibration, offering a scalable alternative to centralized Gaussian process models for multi-agent sensing applications.

• The paper introduces a CRMGP framework that enables decentralized, streaming updates for multi-output Gaussian processes while preserving inter-output covariances.
• It employs matrix-valued kernel recursions and consensus-based fusion to perform efficient local updates with bounded computational complexity.
• Experimental results demonstrate that CRMGP achieves NLPD and RMSE nearly matching centralized MOGP, ensuring accurate field reconstruction and uncertainty estimation.

Consensus-based Recursive Multi-Output Gaussian Processes: Framework and Evaluation

Introduction and Motivation

The paper introduces the Consensus-based Recursive Multi-Output Gaussian Process (CRMGP) framework as a scalable solution for uncertainty-aware learning of vector-valued functions in distributed and streaming environments (2604.10146). Traditional Multi-Output Gaussian Processes (MOGPs) provide rigorous probabilistic modeling for such tasks, particularly via the Linear Model of Coregionalization (LMC). MOGPs can capture cross-output dependencies that are critical in multi-agent systems, environmental monitoring, and spatial field estimation. However, the cubic computational complexity in data size and the centralized nature of classical GP inference render MOGPs impractical for decentralized, resource-constrained, or large-scale streaming contexts.

CRMGP is designed to address three key requirements in multi-agent systems: (i) online, bounded-cost inference accommodating streaming updates, (ii) strict decentralization with only neighbor-to-neighbor data flow—avoiding centralized aggregation and thus any single point of failure, and (iii) preservation of inter-output covariances for improved reconstruction, especially in data-scarce regions.

Technical Framework

CRMGP builds on foundations of recursive GP regression and consensus algorithms to realize a distributed, uncertainty-calibrated, and efficient protocol for multi-output inference.

Each node (agent) maintains a local MOGP that incorporates streaming observations by sequentially updating the posterior over a shared set of basis vectors (inducing points). This is achieved via matrix-valued kernel recursions, adhering to the structure of the LMC. The posterior on these basis points is parameterized in the information form, i.e., via the precision matrix and information vector, affording efficient local updates and additive fusion.

After each local update, nodes execute distributed consensus steps on their information parameters with neighbors over a connected communication graph. The network-wide consensus is realized through weighted averaging (Metropolis or other row-stochastic weights), ensuring agreement on the posterior over the inducing variables without central coordination.

The update protocol guarantees that, upon convergence of the consensus process, each agent arrives at an approximation of the global posterior by appropriately scaling its information parameters. This formulation retains the ability to model cross-output dependencies while delivering parallelizability and bounded computation per agent.

Computational complexity at each update is $\mathcal{O}(\max(D^3, D^2M^2))$ per node (where $D$ is output dimensionality, $M$ is the basis size) for local updates, and $\mathcal{O}(L|\mathcal{N}_i| D^2M^2)$ per node for $L$ consensus steps, which offers substantial efficiency over batch or exact methods for large-scale or streaming data.

Figure 1: Schematic of the Single Output GP (SOGP), highlighting standard inference structure without multi-output correlations.

Figure 2: Schematic representation of SOGP to contrast with multi-output and recursive counterparts in CRMGP.

Experimental Evaluation

The empirical validation uses a synthetic 2D wind field in which wind velocities (U, V) are vector outputs spatially indexed in the plane—characteristic of real-world multi-agent environmental estimation problems.

Figure 3: Illustration of the simulated 2D wind field (U, V components) representing a controlled benchmark for multi-output learning evaluation.

Seven agents, each with spatially distributed and partially overlapping measurement subsets, cooperatively infer the wind field leveraging CRMGP. Comparative benchmarks include: Single-Output GPs (SOGP), full-rank Multi-Output GPs (MOGP), their sparse-inducing approximations (SSOGP, SMOGP), and centralized Recursive MOGP (RMGP).

Performance metrics include negative log predictive density (NLPD), RMSE, and empirical coverage of 95% posterior confidence intervals, directly assessing both predictive accuracy and uncertainty calibration.

Key Numerical Results

• CRMGP achieves NLPD ($-2.71$, $-2.69$ for U, V components) and RMSE ($0.023$) that are extremely close to those of centralized MOGP ($-2.85$, $-2.86$ NLPD and $D$0 RMSE), demonstrating negligible loss in predictive density or calibration despite full decentralization and streaming updates.
• Coverage of confidence intervals is nearly exact (95% for U, 97% for V), confirming robust uncertainty estimation.
• CRMGP thus matches centralized MOGP in reconstruction fidelity and uncertainty quantification while retaining the efficiency and resilience advantages of decentralized operation.

Reconstruction visualizations highlight that CRMGP’s inferred wind field maintains global smoothness and captures wake effects comparably to centralized approaches.

(Figure grid6)

Figure grid6: Reconstruction of the wind field by SOGP, MOGP, SSOGP, SMOGP, RMGP, and CRMGP, demonstrating visually consistent field estimations across methodologies.

Reconstruction error maps further illustrate that, while CRMGP slightly increases error relative to fully centralized baselines, the error is modest and localized.

(Figure grid7)

Figure grid7: Spatial distribution of wind field reconstruction errors by model, showing that CRMGP achieves error rates and spatial error profiles closely paralleling centralized approaches.

Theoretical and Practical Implications

CRMGP provides a rigorous yet resource-efficient means for multi-agent systems to infer vector-valued fields in real time. Unlike classical distributed GP approaches (typically single-output and neglecting covariance structure), CRMGP preserves inter-output coupling via latent mixing (LMC), which is critical for data-sparse or correlated-output regimes. Compared with federated GP or simple averaging schemes, the recursive and consensus formulation directly supports streaming observations and local communication, with bounded memory and computation.

On the theoretical front, CRMGP demonstrates that structured, information-form fusion of local posterior summaries—when embedded in consensus—permits scalable, uncertainty-calibrated inference equivalent to centralized GPs in idealized settings. This claim is substantiated with strong numerical parity in NLPD and calibration results, even with modest basis and agent counts.

CRMGP’s practical impact is significant for multi-robot control, environmental reconstruction, remote sensing, and distributed field estimation, especially when real-time operation, resilience, and uncertainty awareness are non-negotiable.

Future Directions

Future research avenues highlighted by the authors include: (a) integration with distributed optimization and learning algorithms (e.g., ADMM-based hyperparameter tuning), (b) adaptive basis selection to further optimize memory and communication, (c) extension to time-varying or unreliable communication graphs, and (d) empirical deployment on physical multi-agent platforms for closed-loop control.

Conclusion

CRMGP delivers a theoretically sound and empirically validated method for distributed, recursive, and uncertainty-calibrated vector field inference in multi-agent systems. Its design bridges the gap between rigorous probabilistic modeling and practical constraints encountered in decentralized real-world applications, with performance nearly indistinguishable from centralized solutions under typical operational settings. The framework establishes a new standard for scalable, distributed, multi-output Gaussian process inference, with broad applicability and considerable potential for further extension and deployment in complex, sensor-rich environments.