Random Forest Model (RANDM)

• Random Forest Model (RANDM) is an ensemble of randomized decision trees that aggregate predictions via averaging or majority vote for regression and classification.
• Its design-based formulation reveals the role of resampling and feature selection in establishing a structural bias-variance tradeoff and a persistent variance floor.
• Advanced explainability methods using feature contributions and SHAP values enhance understanding of local decision structures in diverse applications.

Searching arXiv for the cited RANDM/random forest papers to ground the article in current arXiv records. Random Forest Model, abbreviated here as RANDM in line with the cited literature, denotes an ensemble of randomized decision trees used for regression and classification. In its canonical form, a training set $\mathcal D_n=\{(X_i,Y_i)\}_{i=1}^n$ is repeatedly resampled, trees are grown with randomized feature selection at internal nodes, and predictions are aggregated by averaging or majority vote. Recent work has also recast random forests as explicit finite-sample statistical designs acting on a fixed dataset, which shifts attention from an algorithmic recipe to the induced weighting, variance, and dependence structure of the ensemble. In parallel, the term RANDM is also used for analytically simplified random-forest schemes, such as centered midpoint-split forests, that support sharp convergence analysis (O'Connell, 13 Feb 2026).

1. Canonical construction and ensemble mechanics

The standard random forest construction begins with a collection of randomized trees built from $\mathcal D_n$. A generic predictor takes the form

$$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$

where each $\Theta_b$ encodes the tree-specific randomization. In the regression/classification formulation emphasized by Mentch and Zhou, $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$, with $\Theta_{\mathcal D,b}$ denoting a bootstrap resample of the data and $\Theta_{mtry,b}$ denoting the random subset of candidate covariates considered at each internal node. If $mtry=p$, every split considers all $p$ variables and the procedure reduces to Bagging; a typical default in regression is $mtry\approx p/3$ (Mentch et al., 2019).

A more detailed algorithmic description specifies the forest by $\mathcal D_n$0. For each tree, one draws either a bootstrap sample or a subsample of size $\mathcal D_n$1, recursively searches over splits only within a randomly drawn set of $\mathcal D_n$2 coordinates, and stops when the current node becomes too small or degenerate. For regression, the prediction at $\mathcal D_n$3 is the average response among training cases in the leaf containing $\mathcal D_n$4; for classification, it is the majority vote in that leaf. Aggregation across trees yields either

$\mathcal D_n$5

for regression or the corresponding majority-vote classifier. In the infinite-forest limit,

$\mathcal D_n$6

where $\mathcal D_n$7 denotes the randomization induced by resampling and feature draws (Biau et al., 2015).

Two classical diagnostic quantities organize much of the literature. The first is the out-of-bag error,

$\mathcal D_n$8

computed by aggregating predictions only from trees in which observation $\mathcal D_n$9 was out of bag. The second is Breiman’s margin for classification, which measures the gap between the vote probability on the true class and the largest competing class probability. The review literature uses these objects to connect empirical performance to tree strength, inter-tree correlation, and hyperparameter selection, especially for $$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$0, $$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$1, tree depth, and leaf size (Biau et al., 2015).

2. Design-based formulation, weighting, and the variance floor

A design-based formulation conditions on a fixed dataset $$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$2 and treats the tree-building randomization $$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$3 as the sole source of variability. In this view, a single tree is a randomized conditional regression function $$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$4, and for a test point $$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$5 the relevant leaf induces a membership set

$$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$6

The tree prediction is then an explicit random weighted average,

$$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$7

A forest of $$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$8 trees is

$$\widehat f_{\mathrm{RF}}(x)=\frac{1}{B}\sum_{b=1}^B T\bigl(x;\mathcal D_n,\Theta_b\bigr),$$9

while the infinite-aggregation target is

$\Theta_b$0

a deterministic function of $\Theta_b$1 alone (O'Connell, 13 Feb 2026).

This framing yields an exact finite-sample variance identity. Writing

$\Theta_b$2

for two independent draws $\Theta_b$3, Theorem 1 gives

$\Theta_b$4

The term $\Theta_b$5 is the finite-aggregation Monte Carlo component and vanishes as $\Theta_b$6, whereas $\Theta_b$7 is a structural dependence term that persists under infinite aggregation. A common misconception is that predictive variability can be made arbitrarily small simply by growing more trees. The design-based result shows that this is false: increasing $\Theta_b$8 removes only the Monte Carlo term, not the dependence floor (O'Connell, 13 Feb 2026).

The same paper further decomposes both $\Theta_b$9 and $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$0 by conditioning on resampling indicators and partitions. For single-tree dispersion, the law of total variance separates variability caused by the resampling of observations from variability caused by partition instability after the included observations are fixed. For inter-tree covariance, the law of total covariance isolates two fundamental mechanisms: reuse of the same training observations across trees and alignment of data-adaptive partitions near $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$1. The shared-observation term is nonnegative and monotone in joint inclusion probabilities, while the alignment term remains even when two trees average disjoint observations from the same covariate region. Theorem 2 shows that if any observation has positive conditional outcome variance and positive probability of receiving nonzero weight at $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$2, then $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$3, establishing a strict positive variance floor under infinite aggregation (O'Connell, 13 Feb 2026).

The design is parameterized in that framework by $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$4, where $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$5 is the fraction of observations used per tree, $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$6 is the number of candidate variables at each split, $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$7 is the minimum terminal node size, and $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$8 is the number of trees. Increasing $\Theta_b=(\Theta_{\mathcal D,b},\Theta_{mtry,b})$9 improves resolution but also increases the reuse component of covariance; decreasing $\Theta_{\mathcal D,b}$0 injects more split randomness and weakens partition alignment but coarsens resolution; larger $\Theta_{\mathcal D,b}$1 enforces coarser averaging and more stable partitions; and $\Theta_{\mathcal D,b}$2 affects only the Monte Carlo term. This suggests a structural bias-variance-resolution tradeoff rather than a one-dimensional “more trees is better” rule (O'Connell, 13 Feb 2026).

3. Randomization as regularization and asymptotic theory

A major theoretical line interprets randomization, especially $\Theta_{\mathcal D,b}$3, as implicit regularization. Under the model

$\Theta_{\mathcal D,b}$4

the degrees of freedom of an estimator $\Theta_{\mathcal D,b}$5 are defined by

$\Theta_{\mathcal D,b}$6

Mentch and Zhou report Monte Carlo evidence that, for fixed tree depth, $\Theta_{\mathcal D,b}$7 increases with $\Theta_{\mathcal D,b}$8, so stronger randomization through smaller $\Theta_{\mathcal D,b}$9 reduces model complexity. In an orthogonal linear model, averaging ordinary least-squares fits over random subspaces of size $\Theta_{mtry,b}$0 yields

$\Theta_{mtry,b}$1

as $\Theta_{mtry,b}$2, which is equivalent to a ridge-type shrinkage factor with

$\Theta_{mtry,b}$3

Their empirical study further shows that $\Theta_{mtry,b}$4 is large when SNR is low, decreases to zero as SNR grows, and eventually reverses sign at high SNR, while the optimal $\Theta_{mtry,b}$5 selected by cross-validation grows almost linearly with SNR. The authors explicitly argue that random forest is not “magically better” than Bagging but is a variance-reduction tool whose variance-bias tradeoff is controlled by $\Theta_{mtry,b}$6 (Mentch et al., 2019).

A separate asymptotic line analyzes subsampled forests built from honest, regular trees. In that setting, each tree is trained on a subset of size $\Theta_{mtry,b}$7 drawn without replacement, splits are chosen with a regularity condition that no coordinate is ignored, and honesty enforces disjoint subsets for split selection and leaf prediction. If $\Theta_{mtry,b}$8 and

$\Theta_{mtry,b}$9

then for a fixed test point $mtry=p$0 the infinite forest $mtry=p$1 is consistent and satisfies a central limit theorem: $mtry=p$2 Moreover, the asymptotic variance can be consistently estimated via the infinitesimal jackknife, enabling pointwise confidence intervals and hypothesis tests. This work is one of the main routes by which random forests become tools for statistical inference rather than only predictive devices (Wager, 2014).

Analytically simplified RANDM variants sharpen the nonparametric picture. For the centered random forest model, a coordinate is selected randomly with probabilities $mtry=p$3 and the current cell is split at its midpoint; each root-to-leaf path has exactly $mtry=p$4 splits, so the number of leaves is $mtry=p$5. Under sparsity, with $mtry=p$6 depending only on an unknown subset of size $mtry=p$7, and with split probabilities tuned to the informative coordinates, the mean-squared prediction error satisfies

$mtry=p$8

where $mtry=p$9 is given explicitly in the paper, and the rate is shown to be sharp through a lower bound. A related median-split subsampled forest admits an improved rate $p$0 under the stated construction. These results do not describe the full Breiman algorithm; rather, they isolate the contribution of randomized recursive partitioning in a tractable regime (Klusowski, 2018).

4. Interpretability and local attribution

Although random forests are often treated as black-box models, several decomposition methods expose their local decision structure. For classification forests, a feature-contribution method defines, at each node $p$1 in tree $p$2, the class-proportion vector $p$3, and for a split on feature $p$4 the local increment

$p$5

for the relevant child node $p$6. If an instance $p$7 follows the path $p$8, then the contribution of feature $p$9 in tree $mtry\approx p/3$0 is

$mtry\approx p/3$1

and averaging over trees gives

$mtry\approx p/3$2

The decomposition satisfies

$mtry\approx p/3$3

where $mtry\approx p/3$4 is the average root-node class proportion across trees; for pure leaves, $mtry\approx p/3$5. This yields per-instance explanations, class-specific median contribution patterns, cluster-based “core clusters,” and log-likelihood typicality scores for assessing whether a prediction conforms to learned decision patterns (Palczewska et al., 2013).

The same paper reports that on the Breast Cancer Wisconsin dataset, median contributions highlighted five key features in exact agreement with Gini importance, while instance-level analysis recovered borderline malignant cases whose decisive variables differed from the global ranking. On the Iris dataset, test cases with similar vote fractions exhibited different feature-contribution profiles, and misclassification was linked to deviations from the class-median contribution pattern. The robustness analysis over 100 independently trained forests showed stable median feature contributions and tight per-feature contribution distributions for held-out instances (Palczewska et al., 2013).

A more recent explainability framework applies SHAP values, computed with TreeSHAP, to a random forest atmospheric neutral density model. In this additive decomposition,

$mtry\approx p/3$6

with $mtry\approx p/3$7 and $mtry\approx p/3$8 the Shapley value of feature $mtry\approx p/3$9. In the reported model, global ranking by mean absolute SHAP value over 4,000 points gave $\mathcal D_n$00 first with mean $\mathcal D_n$01, followed by $\mathcal D_n$02 at $\mathcal D_n$03 and SYM-H ranging from $\mathcal D_n$04 in quiet conditions to $\mathcal D_n$05 in large storms. The analysis further found that SYM-H overtakes $\mathcal D_n$06 between the $\mathcal D_n$07 nT bins, motivating a proposed “storm-time” threshold at SYM-H $\mathcal D_n$08 nT. Because this is a specific atmospheric application, its substantive conclusions are domain-specific; the more general point is that tree-based additive explanations can expose how a trained forest partitions influence among physically meaningful drivers (Bard et al., 30 Sep 2025).

5. Distribution-aware and task-specific forest variants

The canonical random forest objective is mean estimation under regression or majority-vote classification, but several variants modify the splitting rule so that the induced forest targets a different conditional object. For bounded responses in $\mathcal D_n$09, the beta-forest models $\mathcal D_n$10 as $\mathcal D_n$11 with

$\mathcal D_n$12

At each node $\mathcal D_n$13, the algorithm estimates $\mathcal D_n$14 from the node sample and evaluates the total beta log-likelihood $\mathcal D_n$15. A candidate split into $\mathcal D_n$16 and $\mathcal D_n$17 is scored by

$\mathcal D_n$18

and the algorithm selects the split maximizing $\mathcal D_n$19. This construction explicitly respects the unit-interval bounds and the heteroscedastic mean-variance relation, in contrast to ordinary MSE-based regression trees, which can oversplit near $\mathcal D_n$20, undersplit near the boundaries, and produce predictions outside $\mathcal D_n$21. In simulations, beta-forest uniformly outperformed classical random forest on raw or transformed responses, especially when $\mathcal D_n$22 is small or many predictors are irrelevant; only an oracle beta regression with the correct interaction terms beat the beta-forest, and gains in predictive log-likelihood of 10–50 points were typical in moderate-dimensional, low-$\mathcal D_n$23 settings (Weinhold et al., 2019).

Distributional random forests generalize the forest from mean estimation to conditional distribution estimation. Trees are grown on subsamples using honesty, but the split criterion is no longer CART variance reduction. Instead, for a candidate split at node $\mathcal D_n$24 into $\mathcal D_n$25 and $\mathcal D_n$26, the method maximizes an MMD-based statistic

$\mathcal D_n$27

with $\mathcal D_n$28 a random-feature approximation to a characteristic kernel. Prediction proceeds through the forest-induced weights

$\mathcal D_n$29

which form a nonnegative kernel-type weighting function summing to one. These weights define the empirical conditional distribution

$\mathcal D_n$30

from which one can estimate conditional means, quantiles, CDFs, covariances, or parametric likelihood targets by plug-in. Across 14 real multivariate-response datasets, DRF had the lowest average negative log predictive density on most, and in additional studies it matched or outperformed specialized quantile and heterogeneous-effect baselines in several settings (Ćevid et al., 2020).

These variants preserve the forest’s adaptive weighting architecture while redefining what a “good” split means. A plausible implication is that much of the methodological flexibility of RANDM derives from the separation between its partitioning mechanism and its terminal-node target.

6. Weighted forests, imbalance handling, and domain-specific augmentations

Standard random forests weight all trees equally, but the weighting literature questions whether this is justified when bootstrap resampling and random feature selection produce trees with different out-of-bag quality. One set of extensions includes optimal weighted random forests based on accuracy, optimal weighted random forests based on AUC, performance-based weighted random forests using normalized OOB accuracies, and stacking-based random forests that treat tree outputs as meta-features. On 25 public binary classification datasets, repeated 50 times with 75%/25% train/test splits, the average test accuracy was 87.61 for baseline RF, 87.68 for the simple performance-based scheme, and 88.12 for stacked RF with an RF meta-learner on binary OOB outputs. The same study also reports that direct accuracy- and AUC-optimization can overfit the OOB signals and on average did not improve accuracy, whereas stacking gave the largest average gain (Shahhosseini et al., 2020).

In heavily imbalanced cybersecurity data, random forests have also been coupled with adaptive resampling. The ADASYN-Random Forest intrusion detection model first synthesizes minority-class examples in harder-to-learn, majority-rich neighborhoods and then trains an RF classifier with 10 trees, maximum depth 40, Gini splitting, and majority voting. On CICIDS 2017, averaged over 50 trials, plain RF achieved Precision $\mathcal D_n$31, Recall $\mathcal D_n$32, F$\mathcal D_n$33, and AUC $\mathcal D_n$34, while ADASYN + RF achieved Precision $\mathcal D_n$35, Recall $\mathcal D_n$36, F$\mathcal D_n$37, and AUC $\mathcal D_n$38. The paper attributes the improvement primarily to enhanced minority-class recall with negligible loss in precision (Chen et al., 2021).

A different augmentation strategy adds domain-specific covariates rather than synthetic samples. In the Sentiment-Augmented Random Forest model, technical indicators extracted from market data are concatenated with a scalar sentiment score from FinGPT, and hyperparameters are tuned by Random Search with 3-fold cross-validation on AUC-ROC. On daily S&P 500, NASDAQ, and Dow Jones data from 2015–2023, reported test accuracies were 0.67, 0.64, and 0.59 for traditional RF, versus 0.78, 0.85, and 0.82 for SARF, corresponding to an average lift of approximately 9.23%. The authors also note that removing sentiment features causes a drop of approximately 9% in accuracy, while aggressive hyperparameter search alone yields approximately 4–5% improvement (Talazadeh et al., 2024).

Taken together, these extensions reinforce a central theme running across the RANDM literature: the ensemble architecture is stable, but the statistical object represented by the forest depends on design choices about resampling, randomization, split criteria, weighting, and feature construction. The most durable theoretical message is that these choices determine both resolution and dependence. The most durable practical message is that the forest should be viewed not as a monolithic black box, but as a family of randomized, weighted estimators whose behavior is governed by the construction that generates them.