UnbiasedGBM: Bias Correction in GBMs & GBDTs

• UnbiasedGBM is a unified methodology that removes estimation bias in both gradient boosting decision trees and geometric Brownian motion frameworks.
• It employs a novel three-split data protocol—using separate subsets for split search, feature selection, and unbiased gain estimation—to ensure zero expectation on noise splits.
• Empirical studies confirm that UnbiasedGBM improves feature selection in high-dimensional setups and enhances the fidelity of financial time-series sample path simulation.

UnbiasedGBM refers to a family of machine learning and statistical methodologies designed to address intrinsic estimation bias arising from standard techniques built upon Gradient Boosting Machines (GBM) or Geometric Brownian Motion (GBM) frameworks. These approaches are unified by the goal of correcting systematic sources of bias in model construction, inference, or generative modeling, to yield interpretable feature importances, reliable statistics, or unbiased sample paths. The two principal research loci for UnbiasedGBM are unbiased gradient boosting decision trees for general-purpose supervised learning (Zhang et al., 2023), and unbiased generative and inference methods for models governed by geometric Brownian motion, particularly in financial time series (Kim et al., 25 Jul 2025, Chada et al., 2018).

1. Biases in Standard GBDT and GBM Models

Standard GBDT implementations such as XGBoost, LightGBM, and CatBoost use an empirical gain function to drive split selection: $$\widetilde{\mathrm{Gain}}(I, \theta) = \widetilde{\mathcal L}(I) - \widetilde{\mathcal L}(I_L) - \widetilde{\mathcal L}(I_R),$$ with $$\widetilde{\mathcal L}(I) = -\frac{1}{2n} \frac{G_I^2}{H_I}$$, $G_I = \sum_{i \in I} g_i$, $H_I = \sum_{i \in I} h_i$ for first and second-order gradients $g_i, h_i$. This estimator is systematically upward-biased: even under the null (uninformative split), $\mathbb{E}[\widetilde{\mathrm{Gain}}] > 0$ (Zhang et al., 2023). Moreover, data reuse bias arises from searching all candidate splits using the same training data, favoring features with more potential cutpoints and confounding feature importance with cardinality and noise. In diffusion-based modeling of GBM, naïve discretizations or incorrect drift choices introduce bias in simulated or inferred sample paths or posterior statistics (Kim et al., 25 Jul 2025, Chada et al., 2018).

2. Unbiased Gain Metrics in Gradient Boosting

UnbiasedGBM remedies bias in split evaluation by leveraging out-of-bag samples at each node. For a candidate split $\theta$ at node $I$, a node-local random partition yields three sets:

• $\mathcal D$ for split search,
• $\mathcal D_1'$ for feature selection,
• $\mathcal D_2'$ for unbiased gain estimation.

The unbiased post-split loss is estimated as: $$\widetilde{\mathcal L}_{\mathrm{ub}}(I) = -\frac{1}{2n} G_I \frac{G_I'}{H_I'}$$ where $G_I$ and $G_I'$ refer to gradient sums on $\mathcal D$ and $\mathcal D'$, respectively. The unbiased gain is then

$$\widetilde{\mathrm{Gain}}_{\mathrm{ub}}(I, \theta) = \widetilde{\mathcal L}_{\mathrm{ub}}(I) - \widetilde{\mathcal L}_{\mathrm{ub}}(I_L) - \widetilde{\mathcal L}_{\mathrm{ub}}(I_R).$$

Theorem 2 in (Zhang et al., 2023) shows that if the split $\theta$ is uninformative ($X_j \perp y$), then $$\mathbb{E}_{\mathcal D'}[\widetilde{\mathrm{Gain}}_{\mathrm{ub}}(I, \theta)] = 0$$. This cancels the upward bias of standard gain.

3. Algorithmic Structure of UnbiasedGBM in Tree Construction

Each node in the UnbiasedGBM tree construction follows a 1:1:1 data split protocol:

• On $\mathcal D$, identify for each feature the best split by in-sample gain,
• On $\mathcal D_1'$, score each feature's best split (feature selection),
• On $\mathcal D_2'$, compute the unbiased gain for early-stopping control.

A split is performed only if the best feature according to $\mathcal D_1'$ also yields a strictly positive unbiased gain on $\mathcal D_2'$. This sequential validation removes both spurious gain inflation and feature-cardinality bias. The tree is grown recursively under this protocol until maximal depth or leaf-wise early stopping criteria are met (Zhang et al., 2023).

4. UnbiasedGBM in Score-based Diffusion Modeling

In financial time-series modeling, “UnbiasedGBM” refers to a diffusion-based generative framework whereby the forward SDE is exactly a drift-adjusted geometric Brownian motion: $$\mathrm{d}S_t = \mu_t S_t \mathrm{d}t + \sigma_t S_t \mathrm{d}W_t,$$ with $\mu_t = \frac{1}{2} \sigma_t^2$ so that in log-price coordinates,

$\mathrm{d}X_t = \sigma_t \mathrm{d}W_t,$

realizing a variance-exploding (VE) diffusion. Denoising score matching is applied in log-space, with the reverse-time SDE using neural networks for the score approximation. By setting the drift to null in log-space, the generative process becomes unbiased: no artificial trend is imposed and the forward–reverse SDEs are time-symmetric in expectation (Kim et al., 25 Jul 2025).

5. Theoretical Guarantees and Bias Removal

The key guarantees for UnbiasedGBM are:

• For GBDT, Theorems 1 and 2 (Zhang et al., 2023) establish that standard gain is systematically upward-biased for any split, whereas the unbiased gain estimator has zero expectation on noise splits.
• In the diffusion-based UnbiasedGBM, setting the forward drift to exactly offset the quadratic variation (i.e., $\mu_t = \tfrac{1}{2} \sigma_t^2$) removes systematic bias in log-price trajectories, ensuring unbiased generative reconstruction in expectation conditioned on initial/terminal states (Kim et al., 25 Jul 2025).
• For Bayesian inference in discretized GBM diffusions, randomised multilevel Monte Carlo and particle-marginal MCMC methodology yields unbiased posterior functionals with variance reduction and computational efficiency (Chada et al., 2018).

6. Empirical Findings and Comparative Studies

Empirical evaluation in supervised learning encompassed 60 binary-classification datasets (numeric/categorical; small/medium) (Zhang et al., 2023). UnbiasedGBM achieved the lowest average rank in normalized AUC compared to XGBoost, LightGBM, CatBoost, and ablations. Nemenyi tests confirmed significance ($p<10^{-3}$ vs. XGBoost/LightGBM; $p=0.013$ vs. CatBoost). In high-dimensional feature selection, the Unbiased Gain metric outperformed Gain Importance, Permutation Importance, and SHAP in downstream test AUC, validated by a Nemenyi test ($p\le0.042$). A case study on the QSAR Bio-concentration dataset with a 224-level categorical feature demonstrated that UnbiasedGBM eliminates overfitting to high-cardinality noise.

For the diffusion-based UnbiasedGBM, generated samples from real financial series (e.g., S&P 500) replicated stylized facts: heavy-tailed returns (tail exponent close to data), volatility clustering with power-law decay in $|r_t|$ autocorrelation, and leverage effect manifest as prolonged negative lead-lag correlation between $r_t$ and $r_{t+k}^2$ (Kim et al., 25 Jul 2025).

7. Implementation and Practical Considerations

• The data split for unbiased gain at each node does not increase per-tree asymptotic complexity compared to traditional GBDT (Zhang et al., 2023).
• A 1:1:1 split is recommended, with merging of validation sets in small-data regimes.
• UnbiasedGBM inherits conventional GBDT hyperparameters, with the split gain threshold theoretically set to zero.
• The unbiased data splitting logic is compatible with existing block-structured GBDTs (e.g., XGBoost, LightGBM) with minor modifications.
• In diffusion models, neural architectures incorporate Transformer encoders and residual convolutional blocks, with explicit conditioning on diffusion and absolute time (Kim et al., 25 Jul 2025).
• For unbiased posterior inference in discretized GBM, coarse-level PMMH with randomised multilevel particle filtering achieves optimal computational cost and is readily parallelizable (Chada et al., 2018).

UnbiasedGBM represents a unified methodological advance for eliminating core estimation biases in both deterministic (GBDT) and stochastic (GBM) frameworks, improving interpretability, generalization, and the fidelity of inferential or generative tasks.