Interpretable Machine Learning for Spatial Science: A Lie-Algebraic Kernel for Rotationally Anisotropic Gaussian Processes

Abstract: Many three-dimensional spatial fields are anisotropic, with directions of rapid and slow variation that need not align with the coordinate axes. Standard Gaussian process kernels with Automatic Relevance Determination (ARD) capture only axis-aligned anisotropy, while generic full symmetric positive definite (SPD) metrics can represent rotated anisotropy but do not parameterise principal length-scales and directions directly. We introduce an interpretable rotationally anisotropic GP kernel that parameterises a three-dimensional SPD covariance metric using three principal length-scales and an explicit SO(3) rotation. The rotation is represented by an axis-angle vector and mapped to SO(3) via the Lie-algebra exponential map, giving unconstrained Euclidean coordinates for inference while always inducing a valid SPD metric. The construction spans the same family of three-dimensional SPD covariance metrics as a generic full-SPD parameterisation, but exposes the geometry differently: length-scales and orientation are explicit, interpretable, and directly available for prior specification and posterior summaries. We perform Bayesian inference on these quantities using Markov Chain Monte Carlo (MCMC), and characterise the resulting symmetries and weakly identified regimes. On synthetic data with rotated anisotropy, the posterior recovers the generating metric and improves prediction relative to an axis-aligned ARD baseline, while matching the predictive performance of a generic full SPD baseline. When the ground truth is axis-aligned, posterior mass concentrates near the identity rotation and predictive performance matches ARD. On a material-density dataset from a laboratory-fabricated nano-brick, the inferred metric reveals rotated anisotropy that is not captured by axis-aligned kernels.

• The paper introduces a rotationally anisotropic GP kernel using Lie algebra, enabling explicit recovery of principal directions and length-scales.
• It demonstrates the kernel’s ability to recover hyperparameters with high precision, achieving misalignments below 2.5° and lower MAE on real datasets.
• MCMC diagnostics confirm efficient Bayesian inference and interpretable parameter recovery, facilitating direct integration of physical prior knowledge.

Interpretable Rotationally Anisotropic Kernels for Gaussian Processes in Spatial Science

Introduction and Motivation

Gaussian Processes (GPs) provide a principled Bayesian framework for nonparametric regression in spatial data analysis, crucial for applications where scientific interpretability is paramount. Standard strategies for introducing anisotropy in GPs, such as Automatic Relevance Determination (ARD), are limited by their axis-aligned construction; they cannot capture rotated anisotropy intrinsic to many three-dimensional spatial fields. Full Symmetric Positive Definite (SPD) parameterizations generalize anisotropy but their parameters are not directly interpretable in terms of principal directions and characteristic length scales.

This work proposes a kernel for rotationally anisotropic GPs that parameterizes the 3D covariance metric explicitly via principal length scales and an orientation, implemented as an $\mathrm{SO}(3)$ rotation using the Lie algebra exponential map. This yields unconstrained Euclidean coordinates suitable for Bayesian inference while ensuring the kernel remains in the family of valid SPD metrics. Inference is performed via MCMC, and the parameterization enables direct interpretability and flexible prior specification over physical quantities of interest.

Model Formulation

In the proposed kernel, the covariance matrix $M(\ell, a)$ is constructed as:

$$M(\ell, a) = R(a)^\top \; \operatorname{diag}(\ell_x^{-2}, \ell_y^{-2}, \ell_z^{-2}) \; R(a)$$

with $\ell = (\ell_x, \ell_y, \ell_z)^\top$ the principal correlation ranges and $R(a) \in \mathrm{SO}(3)$ the rotation matrix parameterized by an axis–angle vector $a \in \mathbb{R}^3$ via the exponential map:

$R(a) = \exp(U(a))$

where $U(a)$ is a skew-symmetric matrix as per Lie theory. The kernel structure supports standard choices (squared exponential, Matérn), and inference is performed over both the length-scales and the rotation parameters.

This parameterization exposes the SPD metric's geometry: the axes and their orientation are explicit, rather than implicit as with Cholesky factorization or unconstrained matrix entries. Priors and proposals in MCMC can thus directly reflect prior physical or domain knowledge about the system being modeled.

Synthetic Experiments: Hyperparameter Recovery and Predictive Accuracy

On synthetic datasets, the kernel’s properties are systematically validated:

• For rotated anisotropic data ($D_1$), the rotational kernel recovers generating hyperparameters (both length-scales and orientation) to high precision—principal directions' misalignment with ground truth is consistently below $2.5^\circ$.
• In axis-aligned scenarios ($M(\ell, a)$0), the model effectively collapses to the ARD special case, with the rotation posterior concentrated near the identity.
• Predictive performance matches the optimal achievable with a generic SPD kernel and substantially exceeds that of ARD in the presence of rotated anisotropy. For axis-aligned cases, all models attain comparable (and optimal) predictive accuracy.

Figure 1: Posterior predictions on the synthetic datasets. In each column, the top panel shows axis-aligned ARD kernel predictions and the bottom panel shows rotational-kernel predictions. Error bars show $M(\ell, a)$1 predictive uncertainty from the closed-form predictive variance.

MCMC trace diagnostics for both length-scale and rotation parameters demonstrate efficient and interpretable exploration of the posterior, with weakly identified rotations only apparent when eigenvalues (ranges) are nearly identical, making the orientation uninformative—a property visible directly in the parameter traces.

Figure 2: MCMC traces on the rotated synthetic dataset $M(\ell, a)$2 after burn-in for the rotational kernel and axis-aligned ARD kernel hyperparameters.

Figure 3: MCMC traces on the axis-aligned synthetic dataset $M(\ell, a)$3 after burn-in for rotational kernel and axis-aligned ARD kernel parameters.

Application to Real Material-Density Data

The model is applied to a nano-brick material-density dataset, where accurate and interpretable recovery of spatial structure is crucial for scientific investigation. Two primary prediction tasks are evaluated, each holding out distinct spatial planes for testing. The rotational kernel consistently learns non-axis-aligned principal directions, achieving lower mean absolute errors than both axis-aligned and generic SPD models, particularly when test locations are farther from training data along rotated axes.

Figure 4: True and predicted material density surfaces for two held-out $M(\ell, a)$4-planes. Left: ground truth; middle: ARD kernel; right: rotational kernel.

Rotation posteriors (expressed as geodesic angles in $M(\ell, a)$5) are concentrated away from zero, confirming the presence of a genuine, interpretable rotational structure in the data that is invisible to an ARD kernel. MCMC trace diagnostics for real data mirror the synthetic results, with efficient mixing over interpretable parameters.

Figure 5: Trace diagnostics for Task 1, held-out plane $M(\ell, a)$6, after burn-in. Top: axis-aligned ARD kernel. Bottom: rotational kernel.

Figure 6: Trace diagnostics for Task 2, held-out plane $M(\ell, a)$7, after burn-in. Top: axis-aligned ARD kernel. Bottom: rotational kernel.

In a random multi-plane hold-out experiment, the model achieves consistent gains across all test planes, with lower MAE than ARD or SPD models at each slice, demonstrating that improved performance is systematic rather than an artifact of favorable test geometry.

Figure 7: Surface and contour plots at test locations in the $M(\ell, a)$8 plane. Left: true density. Middle: axis-aligned kernel. Right: rotational kernel.

Theoretical and Practical Implications

The proposed parameterization induces no expansion of covariance family expressivity compared to a generic SPD parameterization in three dimensions. However, it fundamentally alters model interpretability, parameter identifiability, and practical inference workflows. Direct recovery of principal correlation directions and ranges allows for:

• More informative prior specification and hypothesis testing with scientific quantities directly,
• Easier diagnostics and understanding of nonidentifiability (e.g., weak orientation identification only when ranges are degenerate),
• Posteriors over interpretable structures, not just matrix entries.

In practical terms, these properties facilitate scientific deployments where model transparency is required and where the geometry of spatial correlation itself is often an object of study, not just a means of improved prediction.

Future Directions

The parameterization naturally admits generalization—for example, to higher-dimensional spatial domains or spatial-temporal fields, structured priors over rotations reflecting domain symmetries, and even modeling structured nonstationarities where both ranges and orientations are spatially varying. Computational extensions, including scalable variational approximations for large data, may further broaden practical impact.

Conclusion

This work establishes a GP kernel for rotationally anisotropic fields with parameters that are both statistically and geometrically interpretable. The axis–angle Lie algebraic construction provides an unconstrained parameterization enabling robust Bayesian inference over physically meaningful quantities. Empirical validation demonstrates both accurate recovery of underlying geometry and practical predictive gains in both synthetic and real datasets, with immediate implications for interpretable ML in spatial science and related disciplines.

---

Reference: "Interpretable Machine Learning for Spatial Science: A Lie-Algebraic Kernel for Rotationally Anisotropic Gaussian Processes" (2605.11179)