Feature-Free Regression Kriging (FFRK)

• FFRK is a geostatistical method that models spatial trends by extracting features directly from the data, eliminating the need for external covariates.
• It integrates regression modeling with kriging of residuals to enhance prediction accuracy, as demonstrated by significant improvements in heavy-metal concentration mapping.
• The method addresses nonstationary spatial variability, offering a robust alternative for applications like subsurface contaminant assessment where high-quality covariate data is scarce.

Feature-Free Regression Kriging (FFRK) is a geostatistical spatial interpolation method that constructs a regression-based trend surface for nonstationary spatial data using automatically extracted geospatial features, rather than relying on external explanatory variables. This approach addresses the challenges of acquiring high-quality covariate layers, particularly in applications where such data are unavailable or irrelevant, such as subsurface contaminant mapping. FFRK’s framework integrates automatic feature extraction, regression modeling, and kriging of residuals to enhance prediction accuracy and generalization for spatial variables with complex, nonstationary patterns (Luo et al., 10 Jul 2025).

1. Mathematical Framework

Let $Z(s)$ denote the spatial variable of interest at location $s\in\mathbb{R}^2$. The FFRK method decomposes $Z(s)$ as

$Z(s) = m(s) + \varepsilon(s)$

where $m(s)$ is the large-scale trend, and $\varepsilon(s)$ is the residual assumed to be second-order stationary. In FFRK, $m(s)$ is modeled entirely by features extracted from the spatial pattern of $Z$ itself, not from external covariates.

For any location $s_p$, a feature vector

$$\mathbf{F}(s_p) = \Bigl[F_{\mathrm{IDW}}(s_p),\, \mathbf{F}_{\mathrm{SVD}}(s_p),\, F_{\mathrm{GOS}}(s_p)\Bigr] \in \mathbb{R}^d$$

is defined, where:

• $s\in\mathbb{R}^2$0: the inverse-distance–weighted mean of the $s\in\mathbb{R}^2$1 nearest $s\in\mathbb{R}^2$2,
• $s\in\mathbb{R}^2$3: a vector of quantiles of $s\in\mathbb{R}^2$4 for the same $s\in\mathbb{R}^2$5 nearest neighbors, capturing local heterogeneity,
• $s\in\mathbb{R}^2$6: a weighted mean using geosimilarity in distributional space.

These features are passed to a regression model (e.g., linear regression, decision tree, random forest, or SVM), yielding the trend

$s\in\mathbb{R}^2$7

The residuals $s\in\mathbb{R}^2$8 are used to estimate a semivariogram and fit a parametric model (e.g., exponential, spherical, or Gaussian). Standard ordinary kriging is then used to interpolate the residuals at prediction sites, and final predictions are given by

$s\in\mathbb{R}^2$9

2. Algorithmic Procedure

The FFRK algorithm involves the following steps:

1. Geofeature Extraction:
   • For each observation and prediction location, identify the set of $Z(s)$0 nearest neighbors.
   • Compute $Z(s)$1 as the inverse-distance–weighted mean.
   • Compute $Z(s)$2 as empirical quantiles over the $Z(s)$3 nearest $Z(s)$4.
   • Compute $Z(s)$5 by weighting neighbors according to distributional similarity using a threshold $Z(s)$6.
2. Trend Regression:
   • Assemble feature vectors into a design matrix $Z(s)$7.
   • Fit the regression model $Z(s)$8 by minimizing squared error using the training samples.
3. Residual Estimation and Kriging:
   • Calculate the residuals from the trend surface.
   • Compute the empirical semivariogram $Z(s)$9.
   • Fit a suitable parametric variogram model.
   • Solve the kriging system to obtain weights for residual interpolation at each prediction site.
4. Prediction:
   • For any location $Z(s) = m(s) + \varepsilon(s)$0, compute the predicted mean as $Z(s) = m(s) + \varepsilon(s)$1.
   • Interpolate residuals $Z(s) = m(s) + \varepsilon(s)$2 via kriging.
   • Output the final prediction as $Z(s) = m(s) + \varepsilon(s)$3.

3. Experimental Evaluation

FFRK was evaluated on surface-soil heavy-metal (Cu, Zn, Pb) concentrations in a Western Australian mining region. The comparative design included:

• 17 baseline methods: ordinary kriging (OK), stratified kriging (StK), four standalone ML models (LM, DT, RF, SVM) using nine external covariates, stratified variants of ML and StK, and regression kriging (RK) with each ML regression.
• FFRK variants (FFRK(LM), FFRK(DT), FFRK(RF), FFRK(SVM)), distinguished by the regression model type and using only extracted geofeatures (no external covariates).
• Hyperparameters: $Z(s) = m(s) + \varepsilon(s)$4, quantile interval 5%.
• Model selection and performance validation by 10-fold cross-validation, using RMSE minimization.

4. Quantitative Results

Performance metrics included cross-validated $Z(s) = m(s) + \varepsilon(s)$5, RMSE, and MAE, averaged over folds. The FFRK variants uniformly outperformed both kriging and standard ML-based regression kriging. For example, for Cu concentration:

Method	$Z(s) = m(s) + \varepsilon(s)$6	RMSE$Z(s) = m(s) + \varepsilon(s)$7	MAE$Z(s) = m(s) + \varepsilon(s)$8
OK	0.27	1.02	0.78
LM	0.31	0.89	0.67
RK(LM)	0.40	0.74	0.55
FFRK(LM)	0.69	0.52	0.39

For Zn and Pb, FFRK achieved 20–70% higher $Z(s) = m(s) + \varepsilon(s)$9 than RK with corresponding ML regression and substantially lower RMSE and MAE. Stratified models provided modest gains but consistently underperformed compared to FFRK. All improvements were statistically significant ($m(s)$0, paired t-test on fold errors) (Luo et al., 10 Jul 2025).

5. Interpretation and Comparative Assessment

The automatically constructed geofeatures—local dependence (IDW), heterogeneity (SVD quantile vector), and global similarity (GOS)—capture essential spatial signals typically sourced from external environmental variables. The viability of FFRK is particularly demonstrated in contexts where covariate layers are unavailable, costly, or physically irrelevant to the spatial process. Empirical evidence confirms that FFRK surpasses:

• Classical kriging (e.g., OK, StK), which assumes stationarity,
• Machine learning models reliant on external covariates,
• Regression kriging with explicit covariates,
• Stratified approaches, which are limited by data fragmentation and discontinuity at class boundaries.

The principal strengths of FFRK include the absence of a need for external covariates, improved accommodation of spatial nonstationarity, and strong predictive accuracy (with $m(s)$1 up to 0.7–0.8). Notable limitations are the requirement for moderately dense spatial sampling for stable geofeature computation and the reliance on hand-crafted feature definitions, which may not capture all relevant geoprocesses. FFRK, in its standard form, is not directly extensible to spatio-temporal or dynamic data without redesigning feature representations.

6. Prospects and Extensions

Future work on FFRK may include:

• Automated feature learning via techniques such as graph-based or deep neural embeddings of the spatial graph, which could generalize the hand-crafted geofeature paradigm.
• Incorporation of external covariates into a “generalized regression kriging” (GRK) framework, where combined use of covariates and geofeatures has empirically produced further gains.
• Spatio-temporal generalizations of FFRK to model time-varying spatial dependencies and local heterogeneity, requiring the design of dynamic geofeatures.

These research directions suggest that the development of feature extraction methods—either hand-crafted or automated—constitutes a critical frontier for nonstationary spatial interpolation, especially in data-poor environments where explanatory variables are inaccessible (Luo et al., 10 Jul 2025).