GWRBoost: Geographically Weighted XGBoost

• GWRBoost is a spatially adaptive ensemble that combines locally weighted regressions through stagewise boosting to model nonstationary spatial relationships.
• It iteratively updates local models using pseudo-residuals and spatial kernels, enhancing flexibility and reducing underfitting compared to classical one-step GWR.
• Empirical studies show GWRBoost improves metrics like RMSE, AICc, and R² while preserving interpretability through analytical model complexity and hat matrix tracking.

Geographically Weighted XGBoost (GW-XGBoost), referred to as GWRBoost in published literature, generalizes the framework of geographically weighted regression (GWR) using a stagewise gradient boosting approach for the estimation of spatially-varying relationships. Unlike classical GWR, which fits spatially localized linear regressions in a single step, GWRBoost iteratively fits and combines multiple locally weighted linear regressions at each geographic location, thereby improving model flexibility and reducing underfitting without sacrificing interpretability. This approach is particularly well-suited for spatial data where underlying relationships between predictors and responses exhibit nonstationarity. Through analytical tracking of model complexity via the hat matrix, GWRBoost maintains quantitative explainability for spatial variation, enabling rigorous model comparison using criteria such as AIC and AICc (Wang et al., 2022).

1. Model Architecture and Loss Objective

GWRBoost operates on a data set $\{(x_i, y_i, s_i)\}_{i=1}^N$, where $x_i \in \mathbb{R}^p$ is the predictor vector for observation $i$, $y_i \in \mathbb{R}$ the response, and $s_i=(u_i, v_i)$ the spatial location. The underlying model is an additive ensemble:

$F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$

with each $f_m(\cdot)$ a localized weighted linear regression (GWR) centered at $s_i$. Fitting is accomplished by minimizing the globally weighted squared-error loss,

$$L(\beta) = \sum_{i=1}^N w_i(s_i)\; \ell(y_i, F^M(x_i)),$$

where $\ell(y, f) = \frac{1}{2}(y-f)^2$ and weights $x_i \in \mathbb{R}^p$0 use a spatial kernel $x_i \in \mathbb{R}^p$1 and bandwidth $x_i \in \mathbb{R}^p$2. Typical kernels include the bisquare $x_i \in \mathbb{R}^p$3 for $x_i \in \mathbb{R}^p$4 and Gaussian $x_i \in \mathbb{R}^p$5.

2. Stagewise Gradient Boosting with Spatial Locality

At each boosting iteration $x_i \in \mathbb{R}^p$6, GWRBoost performs the following steps for each location $x_i \in \mathbb{R}^p$7:

• Residual Computation: Calculate the pseudo-residual

$x_i \in \mathbb{R}^p$8

• Local Model Fitting: Fit a GWR at $x_i \in \mathbb{R}^p$9 to $i$0 using spatial weights $i$1. The local fit is

$i$2

with $i$3 estimated via weighted least squares.

• Ensemble Update: The ensemble prediction at $i$4 is updated,

$i$5

where $i$6 is a learning rate.

If tree-based base learners are desired (the XGBoost style), the model solves a regularized, weighted tree-boosting objective at each step with additional regularization on tree complexity, though in GWRBoost the default base learners are locally weighted linear regressions.

3. Model Complexity and Model Selection Criteria

To enable comparison with other regression models within the AIC/AICc framework, GWRBoost calculates the effective degrees of freedom as the trace of the overall hat matrix after $i$7 boosting steps:

$i$8

where $i$9 is the hat matrix of a single-step GWR. The predicted response is

$y_i \in \mathbb{R}$0

with

$y_i \in \mathbb{R}$1

Model fit is assessed through the Gaussian log-likelihood, and the information criteria are computed as

$y_i \in \mathbb{R}$2

$y_i \in \mathbb{R}$3

This analytic tractability preserves transparency in model selection and regularization.

4. Kernel, Bandwidth, and Hyperparameter Optimization

Kernel and bandwidth selection critically influence the degree of locality. The two canonical kernel types in GWRBoost are:

Kernel	Weight Formula	Support
Bisquare	$y_i \in \mathbb{R}$4	$y_i \in \mathbb{R}$5
Gaussian	$y_i \in \mathbb{R}$6	$y_i \in \mathbb{R}$7

The bandwidth $y_i \in \mathbb{R}$8 is typically selected by minimizing AICc (or a cross-validation score) using grid or golden-section search. Empirically, GWRBoost benefits from bandwidth inflation, commonly setting $y_i \in \mathbb{R}$9 with $s_i=(u_i, v_i)$0, to counteract local overfitting. Regularization is controlled by the learning rate $s_i=(u_i, v_i)$1 (often $s_i=(u_i, v_i)$2–$s_i=(u_i, v_i)$3) and the number of boosting iterations $s_i=(u_i, v_i)$4, with early stopping determined via validation AICc or $s_i=(u_i, v_i)$5.

5. Algorithmic Workflow

The procedural logic of GWRBoost is as follows:

• Initialization: Set $s_i=(u_i, v_i)$6 for all $s_i=(u_i, v_i)$7.
• Boosting Loop ($s_i=(u_i, v_i)$8 to $s_i=(u_i, v_i)$9): For each location $F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$0:
  1. Compute residuals: $F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$1.
  2. Fit local GWR at $F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$2 to residuals $F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$3, using $F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$4.
  3. Form $F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$5.
  4. Update $F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$6.

• Degrees-of-Freedom Calculation: After boosting, compute $F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$7 and $F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$8.

The central distinction from standard GWR is the ensembling over $F^{M}(x_i) = \sum_{m=1}^{M} f_m(x_i),$9 steps; classical GWR fits a single local model per site.

6. Computational Complexity and Empirical Performance

The computational complexity of GWRBoost is dominated by $f_m(\cdot)$0, with each local GWR requiring $f_m(\cdot)$1 operations, and the hat matrix tracking $f_m(\cdot)$2 storage naively. Simulation experiments on a $f_m(\cdot)$3 grid ($f_m(\cdot)$4) show that GWRBoost achieves 18.3% lower RMSE in coefficient estimation and a 67.3% reduction in AICc (GWRBoost $f_m(\cdot)$5, GWR $f_m(\cdot)$6). Moran's I of residuals is near zero for GWRBoost, indicating negligible spatial autocorrelation, compared to GWR's $f_m(\cdot)$7.

In an empirical study on New York City census blocks ($f_m(\cdot)$8, $f_m(\cdot)$9):

Evaluation Metric	GWRBoost	GWR	Relative Change
RSS	261.5	388.6	$s_i$032.7%
AICc	2437.5	3315.6	$s_i$126.5%
$s_i$2	0.882	0.825	$s_i$36.9%
Moran's $s_i$4 (residuals)	$s_i$5	$s_i$6	Lower: random

These results demonstrate substantial improvements in estimation accuracy, model fit, and residual independence compared to single-step GWR.

7. Comparisons with GWR and XGBoost Paradigms

GWRBoost synthesizes concepts from GWR and gradient boosting. Key distinctions include:

Aspect	GWRBoost	GWR	XGBoost-Style Trees
Base Learner	Local weighted linear regression	Local linear	Decision/regression trees
Model Fitting	Boosting-style (sequential)	One-shot per site	Boosting-style (sequential)
Spatial Weights	At every step	Once	Not standard
Analytical AICc/Hat Matrix	Yes	Yes	Not typically analytical

A salient feature is that GWRBoost achieves near-global optimality for the weighted squared-error/AICc criterion while preserving interpretability through explicit local coefficient estimates. If tree-based learners are substituted for the local regression base learners, the algorithm approaches a spatially weighted variant of XGBoost, but would forfeit the closed-form quantification of spatial relationship variation inherent to the GWR approach (Wang et al., 2022).

In summary, GWRBoost extends the GWR paradigm to higher-dimensional and more complex spatial data, providing state-of-the-art performance in spatial regression tasks while remaining analytically transparent, particularly in model selection and interpretation.