Guided Random Forest (GRAF) Overview

• Guided Random Forest is a family of methods that biases tree split gains using external guidance to prioritize informative features in high-dimensional data.
• It utilizes weighting schemes derived from initial Random Forest importance scores to enhance feature selection while keeping tree construction independent for parallelization.
• Empirical studies on gene-expression datasets demonstrate that GRF can improve classification accuracy by balancing feature reduction with retention of useful data redundancy.

Searching arXiv for the cited Guided Random Forest papers and closely related variants. Guided Random Forest denotes a family of Random-Forest-derived methods in which the tree-building process is biased by guidance information rather than left entirely to standard randomization. In the most widely cited formulation for feature selection, Guided Random Forest (GRF), also called GRAF in some contexts, is a feature-selection variant of Random Forest designed for high-dimensional problems such as gene-expression classification; it uses importance scores from an ordinary Random Forest to reweight split gains and is typically followed by retraining a standard Random Forest on the selected features (Deng, 2013). The same name was later reused for a distinct ensemble based on random hyperplanes and global partitioning rather than feature-importance-weighted split scoring (Gupta et al., 2019). This suggests that the term is best understood through its specific formulation rather than by acronym alone.

1. Terminology and scope

The label “Guided Random Forest” is not unique to a single algorithmic construction. In the 2013 formulation, guidance enters at split scoring through feature-specific weights derived from Random Forest importance, and the method is presented primarily as a feature selection procedure. In the 2019 formulation, guidance refers to global propagation of random hyperplanes across partitions in an oblique-tree ensemble. A later paper, “Feature Importance Guided Random Forest Learning with Simulated Annealing Based Hyperparameter Tuning,” is explicitly positioned as conceptually aligned with GRAF but not as a direct implementation of the canonical split-guided method (Balasubramanian et al., 31 Oct 2025).

Variant	Guidance mechanism	Stated role
GRF / GRAF in the RRF package (Deng, 2013)	Split gain multiplied by feature-specific $\lambda_i$	Feature selection for high-dimensional data
GRAF with global partitioning (Gupta et al., 2019)	Random hyperplanes extended to other partitions	Alternative ensemble classifier
FIGRF (Balasubramanian et al., 31 Oct 2025)	Probability-weighted feature sampling plus SA tuning	GRAF-like RF extension

A common misconception is to treat these formulations as interchangeable. They share the general idea of biasing Random Forest construction toward more informative structure, but they do so at different stages: split scoring, partition propagation, or feature-subspace sampling.

2. Canonical GRF as feature-selection-guided Random Forest

In the 2013 GRF formulation, the core modification is local but systematic. At each tree node, the usual split gain for feature $X_i$ is multiplied by a guidance weight:

$gain_G(X_i) = \lambda_i \, gain(X_i)$

where $gain(X_i)$ is the usual Gini information gain for feature $X_i$. The guidance coefficient is defined as

$\lambda_i = 1-\gamma + \gamma \frac{Imp_i}{Imp^*}$

with $Imp_i$ the importance score of feature $X_i$ from an initial ordinary Random Forest, $Imp^*$ the maximum importance score among all features, $\frac{Imp_i}{Imp^*}\in[0,1]$ the normalized importance, and $X_i$0 a parameter controlling how strongly the importance scores influence split selection (Deng, 2013).

When $X_i$1, which is the value used in the experiments, the rule becomes

$X_i$2

Under this setting, features with high Random Forest importance are penalized less and features with low importance are penalized more. If $X_i$3, then $X_i$4 for all features and GRF reduces to ordinary Random Forest.

The stated motivation is twofold. First, gene datasets can contain thousands of features, making ordinary Random Forest hard to interpret and unnecessarily broad in the variables it effectively relies on. Second, the earlier guided regularized random forest (GRRF), while effective for feature selection, builds trees sequentially; features used in earlier trees affect later trees, which induces dependence and makes the method hard to parallelize. GRF was introduced as a simpler guided method that still uses Random Forest importance but keeps tree construction independent.

The paper also notes that the guidance coefficients do not need to come exclusively from Random Forest importance. By specifying $X_i$5, the method can also be guided by other methods such as human insights, and the text explicitly mentions F-scores and other external relevance measures as possible sources. This suggests that GRF is a general guided splitting framework whose canonical instantiation uses Random Forest importance.

3. Relation to ordinary Random Forest and GRRF

Ordinary Random Forest and GRF share the same ensemble-level architecture in one crucial respect: trees are built independently. The difference is that ordinary Random Forest chooses candidate splits by maximizing standard impurity reduction, whereas GRF uses importance-weighted split gains. Because of this weighting, features with higher prior importance are more likely to be selected in future splits, and the resulting model produces a smaller feature subset than ordinary Random Forest (Deng, 2013).

The distinction between GRF and GRRF is sharper. GRRF has sequential construction, and the penalty for a feature depends not only on its Random Forest importance but also on whether it has already been used in earlier trees. It therefore encourages features that are both relevant and non-redundant. GRF, by contrast, builds trees independently, with no dependence on features chosen by earlier trees. It therefore encourages features that are relevant, but not necessarily non-redundant.

This structural difference explains the empirical trade-off emphasized in the paper. GRF selects more features than GRRF, yet it leads to better classification accuracy. The interpretation offered in the source material is that GRRF is more aggressive in compressing the feature set, whereas GRF is less restrictive and retains more potentially useful variables. A plausible implication is that redundancy control and predictive performance need not move in the same direction in very high-dimensional problems.

Tree independence is also the basis for GRF’s computational positioning. Since no tree needs to wait for another to finish and no later tree depends on earlier tree-selected features, GRF is naturally parallelizable. The paper presents this as one of its central advantages over GRRF.

4. Training procedure and implementation in the RRF package

The practical workflow described for GRF is explicit. An ordinary Random Forest is first trained on the full training set to obtain feature importance scores $X_i$6. These scores are then normalized by dividing each score by the maximum importance $X_i$7. Guidance coefficients are computed, and with $X_i$8 the paper uses $X_i$9. A GRF is then built by choosing splits according to $gain_G(X_i) = \lambda_i \, gain(X_i)$0. The selected features are often passed to a fresh Random Forest, producing the GRF-RF pipeline (Deng, 2013).

The implementation example uses the RRF package. The package is loaded with library(RRF). Ordinary Random Forest behavior is obtained with RRF(..., flagReg=0). Importance values are taken from RF$importance[,&quot;MeanDecreaseGini&quot;]</code>, normalized via <code>impRF &lt;- imp/max(imp)</code>, and converted into feature-specific coefficients by <code>coefReg &lt;- (1-gamma) + gamma*impRF</code>. GRF is then built as</p> <p>$gain_G(X_i) = \lambda_i \, gain(X_i)$4

In this implementation, flagReg=0 means the method behaves as RF-like splitting rather than the regularized Random Forest mode, and coefReg supplies the feature-specific guidance weights, i.e. the $gain_G(X_i) = \lambda_i \, gain(X_i)$1 terms.

The worked example in the source material illustrates the intended use pattern rather than only the internal mechanics: ordinary Random Forest used all 500 features, GRF reduced this to 196 features, and Random Forest retrained on those features misclassified fewer test instances. The paper presents this as evidence that GRF is not merely a classifier variant but a feature selection method that can improve downstream Random Forest performance.

5. Empirical evaluation on high-dimensional gene-expression data

The main experimental study evaluates GRF on 10 high-dimensional gene-expression datasets: adenocarcinoma, brain, breast.2.class, breast.3.class, colon, leukemia, lymphoma, nci, prostate, and srbct (Deng, 2013). These datasets are characterized as having very small sample sizes, many classes in some cases, and thousands of features. The paper explicitly lists, for example, adenocarcinoma with 76 instances and 9868 features, brain with 42 instances and 5597 features, leukemia with 38 instances and 3051 features, and nci with 61 instances and 5244 features.

The evaluation protocol uses 100 random train/test splits with a training/testing ratio of 2:1 and 1000 trees per forest. The comparison methods are RF, GRF, GRF-RF, GRRF, and GRRF-RF. Significance testing is performed by paired t-test at the 0.05 level. The text notes that results for RF and GRRF-RF may differ slightly from earlier work because of randomness.

The principal finding is that GRF-RF is best overall among the compared methods. From Table 1, GRF-RF outperforms plain RF on 9 of the 10 datasets, and 7 of those 9 improvements are statistically significant. GRF-RF also outperforms GRRF and GRRF-RF clearly, and it usually outperforms GRF when GRF is used directly as the classifier.

The feature-selection results clarify the methodological trade-off. GRF selects fewer features than RF and generally far fewer than the original full feature set, but it selects more features than GRRF. Despite this, it yields better classification accuracy. The paper’s empirical conclusion is therefore that some redundancy may be acceptable if it improves predictive performance. In the paper’s own framing, if maximal compactness is the primary objective, GRRF may be preferred; if better predictive accuracy with still substantial feature reduction is the objective, GRF is preferable, and GRF-RF provides the best balance of accuracy and interpretability.

6. Alternative GRAF lineage and later guided variants

A distinct 2019 paper uses the name Guided Random Forest (GRAF) for an ensemble classifier built from random hyperplanes and global partitioning rather than feature-importance-weighted split gains (Gupta et al., 2019). In that formulation, each split is not merely local to the partition in which it was generated. A random hyperplane is drawn for the most impure partition, represented by

$gain_G(X_i) = \lambda_i \, gain(X_i)$2

and its effect is extended to other impure partitions whenever doing so improves overall purity. The paper describes this as global partitioning and argues that it bridges the gap between decision trees and boosting algorithms. It also introduces a sensitivity-based data approximation scheme in which points in regions requiring many splits are treated as more informative.

The reported empirical scope of this second GRAF is broad: results are given on 115 UCI datasets, with the claim that GRAF either outperformed or tied the best method on 53 datasets and was better or comparable than AdaBoost, Random Forest, and Gradient Boosting on 94, 69, and 67 datasets, respectively. The paper also presents bias-variance and strength-correlation analyses and argues that global partitioning reduces the generalization error bound.

A later paper, “Feature Importance Guided Random Forest Learning with Simulated Annealing Based Hyperparameter Tuning,” is better understood as a GRAF-like extension rather than as canonical GRF (Balasubramanian et al., 31 Oct 2025). Its method, FIGRF, does not directly modify the split criterion. Instead, it computes a composite feature importance from permutation importance, Gini importance, and mutual information, converts the result into a softmax probability distribution, samples $gain_G(X_i) = \lambda_i \, gain(X_i)$3 features by weighted sampling without replacement for each tree, and tunes n_estimators and max_depth using Simulated Annealing. The paper explicitly states that it is not “GRAF in the strict textbook sense,” because the guidance occurs before tree growth at the feature-subset selection stage rather than inside split optimization. It also notes a limitation: the study does not present a clean ablation isolating the effects of guided feature sampling and SA tuning.

Taken together, these later papers show that “Guided Random Forest” has evolved into a broader design motif rather than a single stabilized algorithmic object. The unifying idea is to make Random-Forest-like ensembles more aware of feature relevance or partition structure, but the specific intervention point—split gain, partition propagation, or feature-subspace sampling—varies across formulations.