Latent Gaussian Models (LGMs)

• Latent Gaussian Models (LGMs) are hierarchical probabilistic models that use latent multivariate Gaussian priors to capture complex dependencies in non-Gaussian observed data.
• They employ diverse inference techniques such as INLA, MCMC, and variational methods to efficiently handle high-dimensional latent fields and non-conjugate likelihoods.
• LGMs extend to various applications including spatial analysis, temporal forecasting, and boosting frameworks, offering scalable solutions and robust uncertainty quantification.

Latent Gaussian Models (LGMs) are a central class of hierarchical probabilistic models in modern statistics and machine learning, characterized by the presence of latent (unobserved) random vectors endowed with multivariate Gaussian priors, upon which the observed data are conditionally dependent through generally non-Gaussian likelihoods. This paradigm underpins a diverse set of methodologies, from Kriging and Gaussian processes in geostatistics and machine learning, to structured mixed effects models, spatial and spatiotemporal inference, Bayesian nonparametric regression, and modern boosting frameworks.

1. Mathematical Definition and Hierarchical Structure

An LGM is defined through a three-level hierarchical Bayesian framework:

1. Data model (observation level):

$$y_i \mid x,\, \theta_1 \sim p\big(y_i \,|\, x_i,\, \theta_1\big), \quad i=1,\dots,n,$$

where $y = (y_1, \ldots, y_n)$ are observed data; $x$ denotes the latent field; $\theta_1$ is a set of hyperparameters.

2. Latent field (process level):

$$x \mid \theta_2 \sim \mathcal{N}(0, Q(\theta_2)^{-1}),$$

with $Q(\theta_2)$ a precision matrix possibly encoding conditional independencies (Gaussian Markov random field structure). The field may be built by the addition of multiple Gaussian components, e.g., fixed effects, spline smoothers, spatial fields, and random effects.

3. Hyperparameter model:

$\theta = (\theta_1, \theta_2) \sim p(\theta)$

The full joint posterior is then

$$p(x, \theta \mid y) \propto \prod_i p(y_i \mid x_i, \theta_1)\, p(x \mid \theta_2)\, p(\theta)$$

This structure generalizes to vector-valued fields, multivariate links, and arbitrary regression or random effect architectures, accommodating both continuous and discrete data types (Stringer et al., 2021, Cabral et al., 2023, Ferrari et al., 27 Jan 2025).

2. Inference Methodologies for LGMs

Bayesian inference for LGMs is technically challenging due to high-dimensional latent fields and non-conjugate likelihoods. The primary inferential methodologies include:

• Laplace and Integrated Nested Laplace Approximation (INLA):

Analytical Laplace approximation for the marginal posterior over hyperparameters $\theta$ by integrating out the latent field $x$ with a Gaussian approximation local to its mode, followed by deterministic or numerical integration over $\theta$ (Stringer et al., 2021).

• Max-and-Smooth (two-step Gaussianization):

For high dimensions or complex data-level likelihoods, each groupwise or sitewise likelihood is first locally approximated by a Gaussian (via Laplace or moment-matching), then inference for the high-dimensional Gaussianized latent field is performed using Gaussian conjugacy and sparse matrix methods. The hyperparameters are sampled via block Gibbs or simple Metropolis–Hastings (Hazra et al., 2021, Hrafnkelsson et al., 2019).

• MCMC and the split sampler:

Blocked MCMC approaches exploit LGM structure by splitting latent variables into data-rich (those appearing in the likelihood) and data-poor blocks. The data-poor block (typically higher-dimensional) is sampled directly from a conditional Gaussian, while the lower-dimensional data-rich block employs Metropolis–Hastings with tailored proposals (Geirsson et al., 2015).

• Variational Bayesian (VB) and dual variational inference:

The variational Gaussian (VG) approximation posits a tractable Gaussian posterior for the latent field and optimizes the evidence lower bound (ELBO). Dual variational inference refines this by formulating and solving a strictly convex dual problem in the space of observation-wise variables, yielding optimal scaling and accelerated convergence (Khan et al., 2013).

• Functional expansion and boosting:

Recent developments (LaGaBoost) embed functional boosting over tree or spline base learners into the LGM structure, jointly learning a non-linear mean function and latent dependence structure. Laplace approximation is used to propagate uncertainty in the boosting steps (Sigrist, 2021).

The complexity and scalability of these methods depend crucially on exploiting sparsity in $Q(\theta_2)$ and the factorization structure of the likelihood. For irregular spatial models (SPDE/FEM meshes), climate and health applications, and generalized joint models, these methods provide substantial computational savings over traditional MCMC (Stringer et al., 2021, Hazra et al., 2021, Niekerk et al., 2019).

3. Model Variants and Extensions

LGMs constitute a broad umbrella encompassing numerous modeling frameworks:

• Latent Map/Latent Variable GPs:

Embedding categorical (qualitative) variables into continuous low-dimensional latent spaces via learned linear mappings, permitting fusion with quantitative variables in any stationary kernel. The learning is performed via joint optimization of the latent-mapping matrix and kernel parameters with a log-marginal likelihood objective. LMGPs, for example, provide neural network interpretations, automatic interaction discovery, and handle variable-length categorical input (Oune et al., 2021).

• Structural Equation Models (SEMs) with Gaussian processes:

Hierarchical DAGs among latent variables, with (possibly non-linear) GP priors on structural functions between latents, and linear-Gaussian measurement models on observables. Sparse inducing-variable approximations are employed for computational efficiency (Silva et al., 2010, Silva et al., 2014).

• Multilevel LGMs for mixed responses:

Two-level models where observed multivariate responses (mixing ordinal and continuous types) are explained via a continuous latent variable that itself follows a spatial GP or GMRF, with suitable probit/threshold links and conjugate priors (Schliep et al., 2012).

• Extended LGMs:

Allowing non-linear or multivariate links between additive predictors and the mean, e.g., multi-resolution spatial fields, partial (non-linear) survival models, or spatial extremes with coupled non-Gaussian latent layers. Theoretical guarantees exist on the asymptotic accuracy of nested Laplace/quadrature schemes in these settings (Stringer et al., 2021).

• Temporal and deep hierarchies:

Deep LGMs for sequence data model time-indexed latent fields and additional explicit state vectors, combining temporal RNN/LSTM transitions with hierarchy in latent Gaussian structure. Training employs the ELBO augmented with custom regularization terms (Johansson et al., 2024).

A tabular summary of selected key model architectures:

Model Type	Latent Layer	Measurement/Link
Standard LGM (INLA)	GMRF	Exponential family
LMGP	Continuous manifold	Stationary kernel GP
GPSEM-LV	DAG-structured GPs	Linear, Gaussian
Multilevel LGM	Spatial GP	Probit/continuous Mix
tDLGM	Deep Gaussian, RNN	Neural, time-aware

4. Practical Considerations: Prior Specification, Variance Partitioning, and Standardization

Prior elicitation in LGMs, especially on variance components, can be challenging due to the lack of interpretability of unconstrained variance parameters. Recent work advocates variance partitioning (VP) approaches, assigning priors to total variance $V$ and proportions $\omega_j$ reflecting effect contributions:

• $$V = \sum_{j=1}^J \sigma_j^2, \qquad \omega_j = \sigma_j^2 / V$$ Standardization procedures ensure that latent model components are scaled so that these parameters reflect true variance contributions. The required steps include: (1) zero-mean constraints for fixed effects, (2) computation of scaling constants using the expectation over the design matrix, with possible modifications for spline or IGMRF structures (Ferrari et al., 27 Jan 2025).

This infrastructure supports practitioners in constructing intuitive priors, e.g., Dirichlet over $\omega_j$, or specifying PC priors on $V$.

5. Model Checking and Robustness

Assessing the appropriateness of the Gaussianity assumption for the latent field is critical but subtle given the hierarchical nature. A general model criticism workflow involves:

• Defining an alternative (non-Gaussian or heavier-tailed) latent prior parametrized by a small perturbation $\eta$.
• Constructing a discrepancy function via the local (score-type) derivative of the log-evidence with respect to $\eta$.
• Performing posterior predictive checks by simulating discrepancy values under the fitted Gaussian model and comparing to the observed value; computing sensitivity measures for key functional summaries (Cabral et al., 2023).

This approach unifies sensitivity analysis and model diagnostics, providing actionable recipes for both routine validation and targeted robustness studies in high-stakes inference.

6. Applications and Empirical Performance

LGMs are applied extensively in spatial epidemiology, climate science, biomedical applications, and materials design. Illustrative results from recent literature:

• Max-and-Smooth for spatial extremes: Efficiently fits high-dimensional extremes models (20 million observations), providing accurate estimation of spatial return levels, uncertainty quantification, and interpretable shrinkage of GEV parameters, outperforming block maxima approaches and traditional MCMC by orders of magnitude in speed (Hazra et al., 2021).
• Latent map GPs for meta-modeling: Enable accurate emulation and Bayesian optimization in mixed categorical/numeric domains, surpassing hand-featurized competitors, with latent embeddings learned directly from data (Oune et al., 2021).
• Latent Gaussian Model Boosting: Outperforms standard boosting and nonparametric regressors in grouped, spatial, and nonlinear regression/classification tasks—especially when nonlinear mean structure and residual dependence coincide (Sigrist, 2021).
• Temporal/deep LGMs (tDLGM): Achieve robust forecasting and imputation under non-stationary noise in sequence modeling benchmarks, maintaining accuracy where LSTMs or vanilla DLGMs degrade (Johansson et al., 2024).

In sum, LGMs form a versatile and computationally tractable foundation for probabilistic modeling in the presence of latent structure, with extensible methodologies covering mixed data types, complex dependencies, and the full spectrum of Bayesian model criticism, robustness, and scalable inference (Stringer et al., 2021, Hazra et al., 2021, Cabral et al., 2023, Ferrari et al., 27 Jan 2025).